annotate hgext/convert/__init__.py @ 5377:756a43a30e34

convert: readd --filemap To handle merges correctly, this revision adds a filemap_source class that wraps a converter_source and does the work necessary to calculate the subgraph we're interested in. The wrapped converter_source must provide a new getchangedfiles method that, given a revision rev, and an index N, returns the list of files that are different in rev and its Nth parent. The implementation depends on the ability to skip some revisions and to change the parents field of the commit objects that we returned earlier. To make the conversion restartable, we assume the revisons in the revmapfile are topologically sorted.
author Alexis S. L. Carvalho <alexis@cecm.usp.br>
date Thu, 04 Oct 2007 23:21:37 -0300
parents d60a067227a5
children 8a2915f57dfc
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
1 # convert.py Foreign SCM converter
3916
645e1dd4b8ae convert-repo: update usage information
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 3911
diff changeset
2 #
4635
63b9d2deed48 Updated copyright notices and add "and others" to "hg version"
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4591
diff changeset
3 # Copyright 2005-2007 Matt Mackall <mpm@selenic.com>
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
4 #
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
5 # This software may be used and distributed according to the terms
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
6 # of the GNU General Public License, incorporated herein by reference.
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
7
5374
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
8 from common import NoRepo, SKIPREV, converter_source, converter_sink
4534
cc9b79216a76 Split convert extension into common and repository type modules
Brendan Cully <brendan@kublai.com>
parents: 4532
diff changeset
9 from cvs import convert_cvs
5355
6b6104430964 convert: support darcs as a source repo
Bryan O'Sullivan <bos@serpentine.com>
parents: 5352
diff changeset
10 from darcs import darcs_source
4534
cc9b79216a76 Split convert extension into common and repository type modules
Brendan Cully <brendan@kublai.com>
parents: 4532
diff changeset
11 from git import convert_git
5013
6c1029aacc9a convert: Support Mercurial as a source, as well as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5012
diff changeset
12 from hg import mercurial_source, mercurial_sink
5112
18abf13064cb Move debugsvnlog to subversion module.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5111
diff changeset
13 from subversion import convert_svn, debugsvnlog
5377
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5376
diff changeset
14 import filemap
4534
cc9b79216a76 Split convert extension into common and repository type modules
Brendan Cully <brendan@kublai.com>
parents: 4532
diff changeset
15
5376
d60a067227a5 convert: move filemapper class to a separate file
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5375
diff changeset
16 import os, shutil
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
17 from mercurial import hg, ui, util, commands
5016
4ebc8693ce72 convert: add filename filtering and renaming support
Bryan O'Sullivan <bos@serpentine.com>
parents: 5014
diff changeset
18 from mercurial.i18n import _
3825
158fce02dc40 Teach convert-repo to deal with mixed charsets in git
Matt Mackall <mpm@selenic.com>
parents: 2649
diff changeset
19
5111
9cda2315c7a9 convert: Use debugsvnlog instead of git-like debug-svn-log.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5107
diff changeset
20 commands.norepo += " convert debugsvnlog"
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
21
5013
6c1029aacc9a convert: Support Mercurial as a source, as well as a sink
Bryan O'Sullivan <bos@serpentine.com>
parents: 5012
diff changeset
22 converters = [convert_cvs, convert_git, convert_svn, mercurial_source,
5355
6b6104430964 convert: support darcs as a source repo
Bryan O'Sullivan <bos@serpentine.com>
parents: 5352
diff changeset
23 mercurial_sink, darcs_source]
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
24
4773
169fe1e6104c convert: make convertsource option handling transparent
Brendan Cully <brendan@kublai.com>
parents: 4759
diff changeset
25 def convertsource(ui, path, **opts):
4756
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
26 for c in converters:
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
27 try:
5018
c7623d2b2a66 convert: get rid of ugly use of hasattr
Bryan O'Sullivan <bos@serpentine.com>
parents: 5016
diff changeset
28 return c.getcommit and c(ui, path, **opts)
c7623d2b2a66 convert: get rid of ugly use of hasattr
Bryan O'Sullivan <bos@serpentine.com>
parents: 5016
diff changeset
29 except (AttributeError, NoRepo):
4756
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
30 pass
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
31 raise util.Abort('%s: unknown repository type' % path)
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
32
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
33 def convertsink(ui, path):
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
34 if not os.path.isdir(path):
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
35 raise util.Abort("%s: not a directory" % path)
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
36 for c in converters:
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
37 try:
5018
c7623d2b2a66 convert: get rid of ugly use of hasattr
Bryan O'Sullivan <bos@serpentine.com>
parents: 5016
diff changeset
38 return c.putcommit and c(ui, path)
c7623d2b2a66 convert: get rid of ugly use of hasattr
Bryan O'Sullivan <bos@serpentine.com>
parents: 5016
diff changeset
39 except (AttributeError, NoRepo):
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
40 pass
4756
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
41 raise util.Abort('%s: unknown repository type' % path)
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
42
5281
a176f9c8b26e convert: rename a class and a function
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5252
diff changeset
43 class converter(object):
5375
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
44 def __init__(self, ui, source, dest, revmapfile, opts):
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
45
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
46 self.source = source
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
47 self.dest = dest
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
48 self.ui = ui
3958
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
49 self.opts = opts
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
50 self.commitcache = {}
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
51 self.revmapfile = revmapfile
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
52 self.revmapfilefd = None
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
53 self.authors = {}
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
54 self.authorfile = None
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
55
5373
6aba1835a7b3 convert: pass the order of the revmapfile to the converter_source
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5355
diff changeset
56 self.maporder = []
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
57 self.map = {}
1655
7bfd4724932a convert-repo: automatically create empty map file
Matt Mackall <mpm@selenic.com>
parents: 1389
diff changeset
58 try:
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
59 origrevmapfile = open(self.revmapfile, 'r')
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
60 for l in origrevmapfile:
1655
7bfd4724932a convert-repo: automatically create empty map file
Matt Mackall <mpm@selenic.com>
parents: 1389
diff changeset
61 sv, dv = l[:-1].split()
5373
6aba1835a7b3 convert: pass the order of the revmapfile to the converter_source
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5355
diff changeset
62 if sv not in self.map:
6aba1835a7b3 convert: pass the order of the revmapfile to the converter_source
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5355
diff changeset
63 self.maporder.append(sv)
1655
7bfd4724932a convert-repo: automatically create empty map file
Matt Mackall <mpm@selenic.com>
parents: 1389
diff changeset
64 self.map[sv] = dv
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
65 origrevmapfile.close()
1655
7bfd4724932a convert-repo: automatically create empty map file
Matt Mackall <mpm@selenic.com>
parents: 1389
diff changeset
66 except IOError:
7bfd4724932a convert-repo: automatically create empty map file
Matt Mackall <mpm@selenic.com>
parents: 1389
diff changeset
67 pass
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
68
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
69 # Read first the dst author map if any
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
70 authorfile = self.dest.authorfile()
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
71 if authorfile and os.path.exists(authorfile):
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
72 self.readauthormap(authorfile)
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
73 # Extend/Override with new author map if necessary
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
74 if opts.get('authors'):
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
75 self.readauthormap(opts.get('authors'))
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
76 self.authorfile = self.dest.authorfile()
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
77
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
78 def walktree(self, heads):
4719
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
79 '''Return a mapping that identifies the uncommitted parents of every
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
80 uncommitted changeset.'''
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
81 visit = heads
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
82 known = {}
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
83 parents = {}
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
84 while visit:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
85 n = visit.pop(0)
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
86 if n in known or n in self.map: continue
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
87 known[n] = 1
5201
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
88 commit = self.cachecommit(n)
4719
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
89 parents[n] = []
5201
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
90 for p in commit.parents:
4719
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
91 parents[n].append(p)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
92 visit.append(p)
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
93
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
94 return parents
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
95
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
96 def toposort(self, parents):
4719
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
97 '''Return an ordering such that every uncommitted changeset is
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
98 preceeded by all its uncommitted ancestors.'''
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
99 visit = parents.keys()
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
100 seen = {}
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
101 children = {}
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
102
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
103 while visit:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
104 n = visit.pop(0)
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
105 if n in seen: continue
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
106 seen[n] = 1
4719
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
107 # Ensure that nodes without parents are present in the 'children'
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
108 # mapping.
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
109 children.setdefault(n, [])
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
110 for p in parents[n]:
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
111 if not p in self.map:
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
112 visit.append(p)
4719
1069205a8894 fix 'convert' with single commit repositories
Hollis Blanchard <hollisb@us.ibm.com>
parents: 4635
diff changeset
113 children.setdefault(p, []).append(n)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
114
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
115 s = []
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
116 removed = {}
692
695dd9a491da convert-repo: deal with packed git and other fixes
mpm@selenic.com
parents: 450
diff changeset
117 visit = children.keys()
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
118 while visit:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
119 n = visit.pop(0)
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
120 if n in removed: continue
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
121 dep = 0
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
122 if n in parents:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
123 for p in parents[n]:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
124 if p in self.map: continue
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
125 if p not in removed:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
126 # we're still dependent
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
127 visit.append(n)
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
128 dep = 1
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
129 break
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
130
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
131 if not dep:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
132 # all n's parents are in the list
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
133 removed[n] = 1
3958
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
134 if n not in self.map:
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
135 s.append(n)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
136 if n in children:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
137 for c in children[n]:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
138 visit.insert(0, c)
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
139
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
140 if self.opts.get('datesort'):
3958
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
141 depth = {}
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
142 for n in s:
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
143 depth[n] = 0
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
144 pl = [p for p in self.commitcache[n].parents
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
145 if p not in self.map]
3958
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
146 if pl:
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
147 depth[n] = max([depth[p] for p in pl]) + 1
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
148
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
149 s = [(depth[n], self.commitcache[n].date, n) for n in s]
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
150 s.sort()
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
151 s = [e[2] for e in s]
2b87d3c5ab8e convert-repo: add option to attempt to sort by date
Matt Mackall <mpm@selenic.com>
parents: 3957
diff changeset
152
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
153 return s
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
154
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
155 def mapentry(self, src, dst):
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
156 if self.revmapfilefd is None:
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
157 try:
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
158 self.revmapfilefd = open(self.revmapfile, "a")
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
159 except IOError, (errno, strerror):
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
160 raise util.Abort("Could not open map file %s: %s, %s\n" % (self.revmapfile, errno, strerror))
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
161 self.map[src] = dst
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
162 self.revmapfilefd.write("%s %s\n" % (src, dst))
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
163 self.revmapfilefd.flush()
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
164
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
165 def writeauthormap(self):
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
166 authorfile = self.authorfile
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
167 if authorfile:
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
168 self.ui.status('Writing author map file %s\n' % authorfile)
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
169 ofile = open(authorfile, 'w+')
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
170 for author in self.authors:
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
171 ofile.write("%s=%s\n" % (author, self.authors[author]))
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
172 ofile.close()
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
173
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
174 def readauthormap(self, authorfile):
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
175 afile = open(authorfile, 'r')
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
176 for line in afile:
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
177 try:
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
178 srcauthor = line.split('=')[0].strip()
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
179 dstauthor = line.split('=')[1].strip()
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
180 if srcauthor in self.authors and dstauthor != self.authors[srcauthor]:
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
181 self.ui.status(
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
182 'Overriding mapping for author %s, was %s, will be %s\n'
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
183 % (srcauthor, self.authors[srcauthor], dstauthor))
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
184 else:
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
185 self.ui.debug('Mapping author %s to %s\n'
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
186 % (srcauthor, dstauthor))
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
187 self.authors[srcauthor] = dstauthor
4590
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
188 except IndexError:
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
189 self.ui.warn(
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
190 'Ignoring bad line in author file map %s: %s\n'
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
191 % (authorfile, line))
80fb4ec512b5 convert: fix various authormap handling bugs
Brendan Cully <brendan@kublai.com>
parents: 4589
diff changeset
192 afile.close()
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
193
5201
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
194 def cachecommit(self, rev):
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
195 commit = self.source.getcommit(rev)
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
196 commit.author = self.authors.get(commit.author, commit.author)
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
197 self.commitcache[rev] = commit
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
198 return commit
653790c2fa52 convert: wrap cached commits author remapping
Patrick Mezard <pmezard@gmail.com>
parents: 5192
diff changeset
199
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
200 def copy(self, rev):
5016
4ebc8693ce72 convert: add filename filtering and renaming support
Bryan O'Sullivan <bos@serpentine.com>
parents: 5014
diff changeset
201 commit = self.commitcache[rev]
4ebc8693ce72 convert: add filename filtering and renaming support
Bryan O'Sullivan <bos@serpentine.com>
parents: 5014
diff changeset
202 do_copies = hasattr(self.dest, 'copyfile')
4ebc8693ce72 convert: add filename filtering and renaming support
Bryan O'Sullivan <bos@serpentine.com>
parents: 5014
diff changeset
203 filenames = []
4939
cdd33a048289 removed trailing whitespace
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4905
diff changeset
204
5374
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
205 changes = self.source.getchanges(rev)
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
206 if isinstance(changes, basestring):
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
207 if changes == SKIPREV:
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
208 dest = SKIPREV
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
209 else:
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
210 dest = self.map[changes]
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
211 self.mapentry(rev, dest)
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
212 return
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
213 files, copies = changes
5172
6b4c332f241b convert: hg: optionally create branches as clones
Brendan Cully <brendan@kublai.com>
parents: 5117
diff changeset
214 parents = [self.map[r] for r in commit.parents]
6b4c332f241b convert: hg: optionally create branches as clones
Brendan Cully <brendan@kublai.com>
parents: 5117
diff changeset
215 if commit.parents:
5202
6f636d13f6b8 convert: load parent commits on-demand
Patrick Mezard <pmezard@gmail.com>
parents: 5201
diff changeset
216 prev = commit.parents[0]
6f636d13f6b8 convert: load parent commits on-demand
Patrick Mezard <pmezard@gmail.com>
parents: 5201
diff changeset
217 if prev not in self.commitcache:
6f636d13f6b8 convert: load parent commits on-demand
Patrick Mezard <pmezard@gmail.com>
parents: 5201
diff changeset
218 self.cachecommit(prev)
6f636d13f6b8 convert: load parent commits on-demand
Patrick Mezard <pmezard@gmail.com>
parents: 5201
diff changeset
219 pbranch = self.commitcache[prev].branch
5172
6b4c332f241b convert: hg: optionally create branches as clones
Brendan Cully <brendan@kublai.com>
parents: 5117
diff changeset
220 else:
6b4c332f241b convert: hg: optionally create branches as clones
Brendan Cully <brendan@kublai.com>
parents: 5117
diff changeset
221 pbranch = None
6b4c332f241b convert: hg: optionally create branches as clones
Brendan Cully <brendan@kublai.com>
parents: 5117
diff changeset
222 self.dest.setbranch(commit.branch, pbranch, parents)
5076
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5018
diff changeset
223 for f, v in files:
5375
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
224 filenames.append(f)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
225 try:
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
226 data = self.source.getfile(f, v)
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
227 except IOError, inst:
5375
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
228 self.dest.delfile(f)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
229 else:
3957
558f52943cd2 convert-repo: add CVS exec bit support
Matt Mackall <mpm@selenic.com>
parents: 3955
diff changeset
230 e = self.source.getmode(f, v)
5375
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
231 self.dest.putfile(f, e, data)
4758
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents: 4756
diff changeset
232 if do_copies:
5076
ef338e34a906 convert: look up copies in getchanges instead of getcommit
Brendan Cully <brendan@kublai.com>
parents: 5018
diff changeset
233 if f in copies:
5375
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
234 copyf = copies[f]
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
235 # Merely marks that a copy happened.
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
236 self.dest.copyfile(copyf, f)
4758
b6a1f2c46c6c convert extension: Add SVN converter
Daniel Holth <dholth@fastmail.fm>
parents: 4756
diff changeset
237
5375
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
238 newnode = self.dest.putcommit(filenames, parents, commit)
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
239 self.mapentry(rev, newnode)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
240
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
241 def convert(self):
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
242 try:
5352
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5281
diff changeset
243 self.source.before()
5014
914054ca532e convert: acquire/release locks periodically
Bryan O'Sullivan <bos@serpentine.com>
parents: 5013
diff changeset
244 self.dest.before()
5373
6aba1835a7b3 convert: pass the order of the revmapfile to the converter_source
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5355
diff changeset
245 self.source.setrevmap(self.map, self.maporder)
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
246 self.ui.status("scanning source...\n")
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
247 heads = self.source.getheads()
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
248 parents = self.walktree(heads)
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
249 self.ui.status("sorting...\n")
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
250 t = self.toposort(parents)
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
251 num = len(t)
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
252 c = None
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
253
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
254 self.ui.status("converting...\n")
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
255 for c in t:
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
256 num -= 1
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
257 desc = self.commitcache[c].desc
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
258 if "\n" in desc:
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
259 desc = desc.splitlines()[0]
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
260 self.ui.status("%d %s\n" % (num, desc))
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
261 self.copy(c)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
262
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
263 tags = self.source.gettags()
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
264 ctags = {}
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
265 for k in tags:
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
266 v = tags[k]
5374
e710874247d1 convert: allow the converter_source to say "skip this revision"
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5373
diff changeset
267 if self.map.get(v, SKIPREV) != SKIPREV:
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
268 ctags[k] = self.map[v]
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
269
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
270 if c and ctags:
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
271 nrev = self.dest.puttags(ctags)
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
272 # write another hash correspondence to override the previous
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
273 # one so we don't end up with extra tag heads
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
274 if nrev:
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
275 self.mapentry(c, nrev)
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
276
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
277 self.writeauthormap()
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
278 finally:
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
279 self.cleanup()
694
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
280
4588
9855939d0c82 convert extension: Save a few opens on the map file
Edouard Gomez <ed.gomez@free.fr>
parents: 4534
diff changeset
281 def cleanup(self):
5352
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5281
diff changeset
282 try:
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5281
diff changeset
283 self.dest.after()
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5281
diff changeset
284 finally:
f0931c0240b4 convert: add before/after hooks for converter sources
Bryan O'Sullivan <bos@serpentine.com>
parents: 5281
diff changeset
285 self.source.after()
5014
914054ca532e convert: acquire/release locks periodically
Bryan O'Sullivan <bos@serpentine.com>
parents: 5013
diff changeset
286 if self.revmapfilefd:
914054ca532e convert: acquire/release locks periodically
Bryan O'Sullivan <bos@serpentine.com>
parents: 5013
diff changeset
287 self.revmapfilefd.close()
694
51eb248d3348 Teach convert-repo about tags
mpm@selenic.com
parents: 692
diff changeset
288
5281
a176f9c8b26e convert: rename a class and a function
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5252
diff changeset
289 def convert(ui, src, dest=None, revmapfile=None, **opts):
4940
71fed370b7a7 Backout ad09ce1d393c and replace ''' with """ to make some highlighting happy.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4939
diff changeset
290 """Convert a foreign SCM repository to a Mercurial one.
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
291
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
292 Accepted source formats:
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
293 - CVS
5355
6b6104430964 convert: support darcs as a source repo
Bryan O'Sullivan <bos@serpentine.com>
parents: 5352
diff changeset
294 - Darcs
6b6104430964 convert: support darcs as a source repo
Bryan O'Sullivan <bos@serpentine.com>
parents: 5352
diff changeset
295 - git
6b6104430964 convert: support darcs as a source repo
Bryan O'Sullivan <bos@serpentine.com>
parents: 5352
diff changeset
296 - Subversion
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
297
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
298 Accepted destination formats:
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
299 - Mercurial
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
300
4753
07efcce17d28 convert: add -r argument specifying latest revision to convert
Brendan Cully <brendan@kublai.com>
parents: 4719
diff changeset
301 If no revision is given, all revisions will be converted. Otherwise,
07efcce17d28 convert: add -r argument specifying latest revision to convert
Brendan Cully <brendan@kublai.com>
parents: 4719
diff changeset
302 convert will only import up to the named revision (given in a format
07efcce17d28 convert: add -r argument specifying latest revision to convert
Brendan Cully <brendan@kublai.com>
parents: 4719
diff changeset
303 understood by the source).
07efcce17d28 convert: add -r argument specifying latest revision to convert
Brendan Cully <brendan@kublai.com>
parents: 4719
diff changeset
304
4884
72ac66e88c43 convert: Use clone's behaviour for the default destionation name.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4719
diff changeset
305 If no destination directory name is specified, it defaults to the
4940
71fed370b7a7 Backout ad09ce1d393c and replace ''' with """ to make some highlighting happy.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4939
diff changeset
306 basename of the source with '-hg' appended. If the destination
71fed370b7a7 Backout ad09ce1d393c and replace ''' with """ to make some highlighting happy.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4939
diff changeset
307 repository doesn't exist, it will be created.
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
308
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
309 If <revmapfile> isn't given, it will be put in a default location
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
310 (<dest>/.hg/shamap by default). The <revmapfile> is a simple text
4884
72ac66e88c43 convert: Use clone's behaviour for the default destionation name.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4719
diff changeset
311 file that maps each source commit ID to the destination ID for
72ac66e88c43 convert: Use clone's behaviour for the default destionation name.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4719
diff changeset
312 that revision, like so:
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
313 <source ID> <destination ID>
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
314
4940
71fed370b7a7 Backout ad09ce1d393c and replace ''' with """ to make some highlighting happy.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4939
diff changeset
315 If the file doesn't exist, it's automatically created. It's updated
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
316 on each commit copied, so convert-repo can be interrupted and can
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
317 be run repeatedly to copy new commits.
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
318
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
319 The [username mapping] file is a simple text file that maps each source
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
320 commit author to a destination commit author. It is handy for source SCMs
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
321 that use unix logins to identify authors (eg: CVS). One line per author
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
322 mapping and the line format is:
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
323 srcauthor=whatever string you want
5252
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
324
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
325 The filemap is a file that allows filtering and remapping of files
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
326 and directories. Comment lines start with '#'. Each line can
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
327 contain one of the following directives:
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
328
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
329 include path/to/file
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
330
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
331 exclude path/to/file
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
332
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
333 rename from/file to/file
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
334
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
335 The 'include' directive causes a file, or all files under a
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
336 directory, to be included in the destination repository. The
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
337 'exclude' directive causes files or directories to be omitted.
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
338 The 'rename' directive renames a file or directory. To rename
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
339 from a subdirectory into the root of the repository, use '.' as
0b0caffcf175 convert: document filemap.
Bryan O'Sullivan <bos@serpentine.com>
parents: 5202
diff changeset
340 the path to rename to.
4940
71fed370b7a7 Backout ad09ce1d393c and replace ''' with """ to make some highlighting happy.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4939
diff changeset
341 """
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
342
4895
fa6c9381d053 convert: manually set encoding to UTF-8
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4884
diff changeset
343 util._encoding = 'UTF-8'
fa6c9381d053 convert: manually set encoding to UTF-8
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 4884
diff changeset
344
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
345 if not dest:
4884
72ac66e88c43 convert: Use clone's behaviour for the default destionation name.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4719
diff changeset
346 dest = hg.defaultdest(src) + "-hg"
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
347 ui.status("assuming destination %s\n" % dest)
4521
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
348
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
349 # Try to be smart and initalize things when required
4754
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
350 created = False
4521
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
351 if os.path.isdir(dest):
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
352 if len(os.listdir(dest)) > 0:
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
353 try:
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
354 hg.repository(ui, dest)
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
355 ui.status("destination %s is a Mercurial repository\n" % dest)
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
356 except hg.RepoError:
4521
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
357 raise util.Abort(
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
358 "destination directory %s is not empty.\n"
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
359 "Please specify an empty directory to be initialized\n"
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
360 "or an already initialized mercurial repository"
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
361 % dest)
4521
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
362 else:
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
363 ui.status("initializing destination %s repository\n" % dest)
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
364 hg.repository(ui, dest, create=True)
4754
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
365 created = True
4521
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
366 elif os.path.exists(dest):
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
367 raise util.Abort("destination %s exists and is not a directory" % dest)
4521
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
368 else:
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
369 ui.status("initializing destination %s repository\n" % dest)
d634b61e9cec Add some more smart when initializing destination repository
Edouard Gomez <ed.gomez@free.fr>
parents: 4520
diff changeset
370 hg.repository(ui, dest, create=True)
4754
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
371 created = True
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
372
4756
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
373 destc = convertsink(ui, dest)
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
374
4754
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
375 try:
4756
8e9d3faec270 convert: split converter into convertsource and convertsink
Brendan Cully <brendan@kublai.com>
parents: 4754
diff changeset
376 srcc = convertsource(ui, src, rev=opts.get('rev'))
4754
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
377 except Exception:
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
378 if created:
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
379 shutil.rmtree(dest, True)
7c8cd400e86a convert: initialize source after destination, cleaning up if source is unusable
Brendan Cully <brendan@kublai.com>
parents: 4753
diff changeset
380 raise
316
c48d069163d6 Add new convert-repo script
mpm@selenic.com
parents:
diff changeset
381
5377
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5376
diff changeset
382 fmap = opts.get('filemap')
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5376
diff changeset
383 if fmap:
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5376
diff changeset
384 srcc = filemap.filemap_source(ui, srcc, fmap)
756a43a30e34 convert: readd --filemap
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5376
diff changeset
385
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
386 if not revmapfile:
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
387 try:
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
388 revmapfile = destc.revmapfile()
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
389 except:
5011
89fbb0a5e8e3 convert: rename mapfile to revmapfile, so we can map more than just revs
Bryan O'Sullivan <bos@serpentine.com>
parents: 4940
diff changeset
390 revmapfile = os.path.join(destc, "map")
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
391
5375
dae323e453aa convert: disable current --filemap support
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5374
diff changeset
392 c = converter(ui, srcc, destc, revmapfile, opts)
3947
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
393 c.convert()
0fab73b3f453 convert-repo: add some smarts
Matt Mackall <mpm@selenic.com>
parents: 3916
diff changeset
394
5100
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5076
diff changeset
395
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
396 cmdtable = {
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
397 "convert":
5281
a176f9c8b26e convert: rename a class and a function
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 5252
diff changeset
398 (convert,
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
399 [('A', 'authors', '', 'username mapping filename'),
5016
4ebc8693ce72 convert: add filename filtering and renaming support
Bryan O'Sullivan <bos@serpentine.com>
parents: 5014
diff changeset
400 ('', 'filemap', '', 'remap file names using contents of file'),
4753
07efcce17d28 convert: add -r argument specifying latest revision to convert
Brendan Cully <brendan@kublai.com>
parents: 4719
diff changeset
401 ('r', 'rev', '', 'import up to target revision REV'),
4589
451e91ed535e convert extension: Add support for username mapping
Edouard Gomez <ed.gomez@free.fr>
parents: 4588
diff changeset
402 ('', 'datesort', None, 'try to sort changesets by date')],
4532
c3a78a49d7f0 Some small cleanups for convert extension:
Thomas Arendsen Hein <thomas@intevation.de>
parents: 4521
diff changeset
403 'hg convert [OPTION]... SOURCE [DEST [MAPFILE]]'),
5111
9cda2315c7a9 convert: Use debugsvnlog instead of git-like debug-svn-log.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5107
diff changeset
404 "debugsvnlog":
5100
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5076
diff changeset
405 (debugsvnlog,
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5076
diff changeset
406 [],
5111
9cda2315c7a9 convert: Use debugsvnlog instead of git-like debug-svn-log.
Thomas Arendsen Hein <thomas@intevation.de>
parents: 5107
diff changeset
407 'hg debugsvnlog'),
4513
ac2fe196ac9b Turns convert.py into a real extension
Edouard Gomez <ed.gomez@free.fr>
parents: 4512
diff changeset
408 }
5100
39b6eaee6fd7 convert: replace fork with subprocess call.
Patrick Mezard <pmezard@gmail.com>
parents: 5076
diff changeset
409