mercurial/httprangereader.py
author Benoit Boissinot <benoit.boissinot@ens-lyon.org>
Thu, 08 Dec 2005 15:12:02 +0100
changeset 1677 11d12bd6e1dc
parent 1559 59b3639df0a9
child 2138 f5046cab9e2e
permissions -rw-r--r--
cleanup of revlog.group when repository is local revlog.group cached every chunk from the revlog, the behaviour was needed to minimize the roundtrip with old-http. The patch export the information that the repository is local or not from the repository object down to the revlog. Then it uses the workaround for old-http only if the repository is non-local. The memory used server side when pulling goes down to less than 30Mo maximum whereas without the patch more than 160Mo was used when cloning the linux kernel repository. The time used by cloning is roughly the same (although some caching could be implemented if needed): before 110.25user 20.90system 2:52.00elapsed 76%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+708707minor)pagefaults 0swaps after 112.85user 22.98system 2:50.66elapsed 79%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (0major+862862minor)pagefaults 0swaps
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
372
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     1
# httprangereader.py - just what it says
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     2
#
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     3
# Copyright 2005 Matt Mackall <mpm@selenic.com>
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     4
#
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     5
# This software may be used and distributed according to the terms
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     6
# of the GNU General Public License, incorporated herein by reference.
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     7
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     8
import byterange, urllib2
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     9
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 372
diff changeset
    10
class httprangereader(object):
372
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    11
    def __init__(self, url):
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    12
        self.url = url
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    13
        self.pos = 0
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    14
    def seek(self, pos):
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    15
        self.pos = pos
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    16
    def read(self, bytes=None):
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    17
        opener = urllib2.build_opener(byterange.HTTPRangeHandler())
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    18
        urllib2.install_opener(opener)
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    19
        req = urllib2.Request(self.url)
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    20
        end = ''
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    21
        if bytes: end = self.pos + bytes
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    22
        req.add_header('Range', 'bytes=%d-%s' % (self.pos, end))
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    23
        f = urllib2.urlopen(req)
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    24
        return f.read()