annotate hgext/convert/git.py @ 5377:756a43a30e34

convert: readd --filemap To handle merges correctly, this revision adds a filemap_source class that wraps a converter_source and does the work necessary to calculate the subgraph we're interested in. The wrapped converter_source must provide a new getchangedfiles method that, given a revision rev, and an index N, returns the list of files that are different in rev and its Nth parent. The implementation depends on the ability to skip some revisions and to change the parents field of the commit objects that we returned earlier. To make the conversion restartable, we assume the revisons in the revmapfile are topologically sorted.
author Alexis S. L. Carvalho <alexis@cecm.usp.br>
date Thu, 04 Oct 2007 23:21:37 -0300
parents 24de027551c1
children a5a7f7fd5554
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4534
cc9b79216a76 Split convert extension into common and repository type modules
Brendan Cully <brendan@kublai.com>
parents: 4532
diff changeset
1 # git support for the convert extension
3825
158fce02dc40 Teach convert-repo to deal with mixed charsets in git
Matt Mackall <mpm@selenic.com>
parents: 2649
diff changeset
2
4534
cc9b79216a76 Split convert extension into common and repository type modules
Brendan Cully <brendan@kublai.com>
parents: 4532
diff changeset
3 import os
5233
9d7052f17d77 convert: fix /dev/null redirections under Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5229
diff changeset
4 from mercurial import util
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
5
4534
cc9b79216a76 Split convert extension into common and repository type modules
Brendan Cully <brendan@kublai.com>
parents: 4532
diff changeset
6 from common import NoRepo, commit, converter_source
3955
9af4b853ed4d convert-repo: add CVS branch support
Matt Mackall <mpm@selenic.com>
parents: 3954
diff changeset
7
4447
af013ae3ca10 use documented convert-repo interface
Daniel Holth <dholth@fastmail.fm>
parents: 4446
diff changeset
8 class convert_git(converter_source):
5229
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
9 # Windows does not support GIT_DIR= construct while other systems
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
10 # cannot remove environment variable. Just assume none have
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
11 # both issues.
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
12 if hasattr(os, 'unsetenv'):
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
13 def gitcmd(self, s):
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
14 prevgitdir = os.environ.get('GIT_DIR')
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
15 os.environ['GIT_DIR'] = self.path
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
16 try:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
17 return os.popen(s)
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
18 finally:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
19 if prevgitdir is None:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
20 del os.environ['GIT_DIR']
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
21 else:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
22 os.environ['GIT_DIR'] = prevgitdir
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
23 else:
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
24 def gitcmd(self, s):
149742a628fd convert: fix issue702 about GIT_DIR= construct unsupported under Windows.
Patrick Mezard <pmezard@gmail.com>
parents: 5228
diff changeset
25 return os.popen('GIT_DIR=%s %s' % (self.path, s))
4760
2d0a823cbba5 convert: gitcmd wrapper for os.popen
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
26
4753
07efcce17d28 convert: add -r argument specifying latest revision to convert
Brendan Cully <brendan@kublai.com>
parents: 4752
diff changeset
27 def __init__(self, ui, path, rev=None):
4806
15a3cbfc6568 convert: call superclass init from engine init functions
Brendan Cully <brendan@kublai.com>
parents: 4761
diff changeset
28 super(convert_git, self).__init__(ui, path, rev=rev)
15a3cbfc6568 convert: call superclass init from engine init functions
Brendan Cully <brendan@kublai.com>
parents: 4761
diff changeset
29
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
30 if os.path.isdir(path + "/.git"):
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
31 path += "/.git"
4111
06d65498f73b convert-repo: use .git/objects/ rather than .git/HEAD to detect git repos
Matt Mackall <mpm@selenic.com>
parents: 4082
diff changeset
32 if not os.path.exists(path + "/objects"):
3954
fad134931327 convert-repo: add basic CVS import support
Matt Mackall <mpm@selenic.com>
parents: 3948
diff changeset
33 raise NoRepo("couldn't open GIT repo %s" % path)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
34 self.path = path
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
35
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
36 def getheads(self):
4761
f52bfe566583 convert: import all branches from git repositories
Brendan Cully <brendan@kublai.com>
parents: 4760
diff changeset
37 if not self.rev:
f52bfe566583 convert: import all branches from git repositories
Brendan Cully <brendan@kublai.com>
parents: 4760
diff changeset
38 return self.gitcmd('git-rev-parse --branches').read().splitlines()
f52bfe566583 convert: import all branches from git repositories
Brendan Cully <brendan@kublai.com>
parents: 4760
diff changeset
39 else:
f52bfe566583 convert: import all branches from git repositories
Brendan Cully <brendan@kublai.com>
parents: 4760
diff changeset
40 fh = self.gitcmd("git-rev-parse --verify %s" % self.rev)
f52bfe566583 convert: import all branches from git repositories
Brendan Cully <brendan@kublai.com>
parents: 4760
diff changeset
41 return [fh.read()[:-1]]
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
42
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
43 def catfile(self, rev, type):
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
44 if rev == "0" * 40: raise IOError()
5233
9d7052f17d77 convert: fix /dev/null redirections under Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5229
diff changeset
45 fh = self.gitcmd("git-cat-file %s %s 2>%s" % (type, rev,
9d7052f17d77 convert: fix /dev/null redirections under Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5229
diff changeset
46 util.nulldev))
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
47 return fh.read()
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
48
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
49 def getfile(self, name, rev):
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
50 return self.catfile(rev, "blob")
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
51
3957
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3955
diff changeset
52 def getmode(self, name, rev):
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3955
diff changeset
53 return self.modecache[(name, rev)]
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3955
diff changeset
54
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
55 def getchanges(self, version):
3957
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3955
diff changeset
56 self.modecache = {}
5228
39e6deaa8b81 convert: gitcmd wrapper for os.popen
Brendan Cully <brendan@kublai.com>
parents: 4721
diff changeset
57 fh = self.gitcmd("git-diff-tree --root -m -r %s" % version)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
58 changes = []
5335
88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5233
diff changeset
59 seen = {}
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
60 for l in fh:
5335
88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5233
diff changeset
61 if "\t" not in l:
88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5233
diff changeset
62 continue
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
63 m, f = l[:-1].split("\t")
5335
88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5233
diff changeset
64 if f in seen:
88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5233
diff changeset
65 continue
88e931f74e8b convert_git: avoid returning two entries for the same file in getchanges
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5233
diff changeset
66 seen[f] = 1
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
67 m = m.split()
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
68 h = m[3]
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
69 p = (m[1] == "100755")
4082
6b2909e84203 convert-repo converts symlinks from git
Daniel Holth <dholth@fastmail.fm>
parents: 4062
diff changeset
70 s = (m[1] == "120000")
6b2909e84203 convert-repo converts symlinks from git
Daniel Holth <dholth@fastmail.fm>
parents: 4062
diff changeset
71 self.modecache[(f, h)] = (p and "x") or (s and "l") or ""
3957
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3955
diff changeset
72 changes.append((f, h))
5076
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 4873
diff changeset
73 return (changes, {})
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
74
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
75 def getcommit(self, version):
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
76 c = self.catfile(version, "commit") # read the commit hash
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
77 end = c.find("\n\n")
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
78 message = c[end+2:]
4752
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4721
diff changeset
79 message = self.recode(message)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
80 l = c[:end].splitlines()
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
81 manifest = l[0].split()[1]
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
82 parents = []
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
83 for e in l[1:]:
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
84 n, v = e.split(" ", 1)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
85 if n == "author":
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
86 p = v.split()
1385
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
87 tm, tz = p[-2:]
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
88 author = " ".join(p[:-2])
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
89 if author[0] == "<": author = author[1:-1]
4752
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4721
diff changeset
90 author = self.recode(author)
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
91 if n == "committer":
431
dfc44f3f587c convert-repo fixups
mpm@selenic.com
parents: 316
diff changeset
92 p = v.split()
1385
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
93 tm, tz = p[-2:]
431
dfc44f3f587c convert-repo fixups
mpm@selenic.com
parents: 316
diff changeset
94 committer = " ".join(p[:-2])
dfc44f3f587c convert-repo fixups
mpm@selenic.com
parents: 316
diff changeset
95 if committer[0] == "<": committer = committer[1:-1]
4752
20ec5cc02f18 convert: ove recode method into converter_source
Brendan Cully <brendan@kublai.com>
parents: 4721
diff changeset
96 committer = self.recode(committer)
3910
4bc5a2405b12 convert-repo: fix recoding of committer
Matt Mackall <mpm@selenic.com>
parents: 3825
diff changeset
97 message += "\ncommitter: %s\n" % committer
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
98 if n == "parent": parents.append(v)
1385
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
99
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
100 tzs, tzh, tzm = tz[-5:-4] + "1", tz[-4:-2], tz[-2:]
2093
5cc414722587 convert-repo: fix reversed time zone offset
Vadim Gelfer <vadim.gelfer@gmail.com>
parents: 1715
diff changeset
101 tz = -int(tzs) * (int(tzh) * 3600 + int(tzm))
1385
adb3de56635b convert-repo: Fix timezone handling
Matt Mackall <mpm@selenic.com>
parents: 1335
diff changeset
102 date = tm + " " + str(tz)
4721
96614af3c679 convert: "unknown" is a string
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4687
diff changeset
103 author = author or "unknown"
3955
9af4b853ed4d convert-repo: add CVS branch support
Matt Mackall <mpm@selenic.com>
parents: 3954
diff changeset
104
4873
28b23b9073a8 convert: record the source revision in the changelog
Brendan Cully <brendan@kublai.com>
parents: 4809
diff changeset
105 c = commit(parents=parents, date=date, author=author, desc=message,
28b23b9073a8 convert: record the source revision in the changelog
Brendan Cully <brendan@kublai.com>
parents: 4809
diff changeset
106 rev=version)
3955
9af4b853ed4d convert-repo: add CVS branch support
Matt Mackall <mpm@selenic.com>
parents: 3954
diff changeset
107 return c
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
108
694
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
109 def gettags(self):
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
110 tags = {}
5233
9d7052f17d77 convert: fix /dev/null redirections under Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5229
diff changeset
111 fh = self.gitcmd('git-ls-remote --tags "%s" 2>%s' % (self.path,
9d7052f17d77 convert: fix /dev/null redirections under Windows
Patrick Mezard <pmezard@gmail.com>
parents: 5229
diff changeset
112 util.nulldev))
4062
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
113 prefix = 'refs/tags/'
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
114 for line in fh:
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
115 line = line.strip()
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
116 if not line.endswith("^{}"):
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
117 continue
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
118 node, tag = line.split(None, 1)
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
119 if not tag.startswith(prefix):
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
120 continue
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
121 tag = tag[len(prefix):-3]
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
122 tags[tag] = node
516f883e3d79 convert-repo: handle packed git tags
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4047
diff changeset
123
694
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
124 return tags