mercurial/httprangereader.py
author Vadim Gelfer <vadim.gelfer@gmail.com>
Fri, 14 Jul 2006 11:17:22 -0700
changeset 2612 ffb895f16925
parent 2161 12e11413ca19
child 2858 345bac2bc4ec
permissions -rw-r--r--
add support for streaming clone. existing clone code uses pull to get changes from remote repo. is very slow, uses lots of memory and cpu. new clone code has server write file data straight to client, client writes file data straight to disk. memory and cpu used are very low, clone is much faster over lan. new client can still clone with pull, can still clone from older servers. new server can still serve older clients.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
372
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     1
# httprangereader.py - just what it says
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     2
#
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     3
# Copyright 2005 Matt Mackall <mpm@selenic.com>
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     4
#
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     5
# This software may be used and distributed according to the terms
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     6
# of the GNU General Public License, incorporated herein by reference.
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     7
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     8
import byterange, urllib2
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
     9
1559
59b3639df0a9 Convert all classes to new-style classes by deriving them from object.
Eric Hopper <hopper@omnifarious.org>
parents: 372
diff changeset
    10
class httprangereader(object):
372
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    11
    def __init__(self, url):
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    12
        self.url = url
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    13
        self.pos = 0
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    14
    def seek(self, pos):
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    15
        self.pos = pos
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    16
    def read(self, bytes=None):
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    17
        opener = urllib2.build_opener(byterange.HTTPRangeHandler())
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    18
        urllib2.install_opener(opener)
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    19
        req = urllib2.Request(self.url)
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    20
        end = ''
2138
f5046cab9e2e Fix revlog-ng interaction with old-http.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 1559
diff changeset
    21
        if bytes:
f5046cab9e2e Fix revlog-ng interaction with old-http.
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 1559
diff changeset
    22
            end = self.pos + bytes - 1
372
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    23
        req.add_header('Range', 'bytes=%d-%s' % (self.pos, end))
4b0f562c61f4 Move httprangereader into its own file
mpm@selenic.com
parents:
diff changeset
    24
        f = urllib2.urlopen(req)
2161
12e11413ca19 Fix just introduced possible old-http bug
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2138
diff changeset
    25
        data = f.read()
12e11413ca19 Fix just introduced possible old-http bug
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2138
diff changeset
    26
        if bytes:
12e11413ca19 Fix just introduced possible old-http bug
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2138
diff changeset
    27
            data = data[:bytes]
12e11413ca19 Fix just introduced possible old-http bug
Alexis S. L. Carvalho <alexis@cecm.usp.br>
parents: 2138
diff changeset
    28
        return data