JDK7 changed behavior slightly on some InputStream types, resulting in
the first read being shorter than the count requested. That caused us
to overwrite the earlier part of the buffer with later data, as the
offset index wasn't updated in the loop.
Fix the loop to increment offset by the number of bytes read in this
iteration, so the next read appends to the buffer rather than doing an
overwrite.
Bug: 338119
Change-Id: I222fb2f993cd9b637b6b8d93daab5777ef7ec7a6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
FetchCommand now does not set a null credentials provider on
Transport because in this case the default provider is replaced with
null and the default mechanism for providing credentials is not
working.
Change-Id: I44096aa856f031545df39d4b09af198caa2c21f6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In bc1af8459e ("RevWalk: Don't reset ObjectReader when stopping") we
stopped releasing the reader when the current log traversal is over.
This should have also been applied to the merge base logic that is
buried within MergeGenerator, but got missed.
Change-Id: I8328f43f02cba06fd545e22134872e781b9d4d36
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Also use FS.resolve() to properly resolve files from path strings.
Bug: 328428 (partial fix)
Change-Id: I41d94694f220dcb85605c9acadfffb1fa23beaeb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
We did not record the time spent on the object reuse search or the
object size lookup, both of which occur between the counting phase and
the compressing phase. If there are enough objects involved, these
times can be significant so its worth timing them and recording it.
Change-Id: I89084acfc598bb6533d75d90cb8de459f0ed93be
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Support for --no-standard-notes and --show-notes=REF options is added
to the Log command. The --show-notes option can be specified more than
once if more than one notes branch should be used for showing notes.
The notes are displayed from note branches in the order how the note
branches are specified in the command line. However, the standard note,
from the refs/notes/commits, is always displayed as first unless
the --no-standard-notes options is given.
Change-Id: I4e7940804ed9d388b625b8e8a8e25bfcf5ee15a6
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
The total delta count is supposed to include reused deltas, not
just newly created deltas.
Change-Id: I98cbdcef80d59714a4f62ff322e7b709b08b6d26
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A checked Exception is thrown instead.
The reason for throwing an Exception is that the state of the
repository is inconsistent in this case: There is a merge
configuration containing a non-existing local branch. Ideally the
deletion of a local branch should also delete the corresponding
merge configuration.
Bug: 337315
Change-Id: I8ed57d5aaed60aaab685fc11a8695e474e60215f
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Some clients coming through proxies may advertise a different
Accept-Encoding, for example "Accept-Encoding: gzip(proxy)".
Matching by substring causes us to identify this as a false positive;
that the client understands gzip encoding and will inflate the
response before reading it.
In this particular case however it doesn't. Its the reverse proxy
server in front of JGit letting us know the proxy<->JGit link can
be gzip compressed, while the client<->proxy part of the link is not:
client <-- no gzip --> proxy <-- gzip --> JGit
Use a more standard method of parsing by splitting the value into
tokens, and only using gzip if one of the tokens is exactly the
string "gzip". Add a unit test to make sure this isn't broken in
the future.
Change-Id: Ib4c40f9db177322c7a2640808a6c10b3c4a73819
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Many source browsers and network related tools like UploadPack need
to find and parse the target of all branches and annotated tags
within the repository during their startup phase. Clustering these
together into the same part of the pack file will improve locality,
reducing thrashing when an application starts and needs to load
all of these into memory at once.
To prevent bottlenecking basic log viewing tools that are scannning
backwards from the tip of a current branch (and don't need tags)
we place this cluster of older targets after 4096 newer commits
have already been placed into the pack stream. 4096 was chosen as
a rough guess, but was based on a few factors:
- log viewers typically show 5-200 commits per page
- users only view the first page or two
- DHT can cram 2200-4000 commits per 1 MiB chunk
thus these will fall into the second commit chunk (roughly)
Unfortunately this placement hurts history tools that are scanning
backwards through the commit graph and completely ignored tags or
branch heads when they started.
An ancient tagged commit is no longer positioned behind its first
child (its now much earlier), resulting in a page fault for the
parser to reload this cluster of objects on demand. This may be
an acceptable loss. If a user is walking backwards and has already
scanned through more than 4096 commits of history, waiting for the
region to reload isn't really that bad compared to the amount of
time already spent.
If the repository is so small that there are less than 4096 commits,
this change has no impact on the placement of objects.
Change-Id: If3052e430d305e17878d94145c93754f56b74c61
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the underlying storage has a high latency per SHA-1 lookup
(e.g. the DHT support we are working on), parsing each wanted
annotated tag object back to its underlying commit is too slow,
its a sequential lookup for each tag. With hundreds of tags in
a repository this takes far too long.
Instead queue up a list of the tags whose objects need to be found,
and then locate all of those in one parseAny batch. This works
for the common case of annotated tag to single tree or commit.
For the less often used tag->tag->commit, it at least gets us
one level parsed in the larger batch before we have to go back to
sequential lookups.
Change-Id: I94beef3f14281406f15c8cf9fa02d83faf102a19
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the CachedPack knows its delta count, we need to increment both
the totalDeltas and reusedDeltas fields of the stats object.
Change-Id: I70113609c22476ce7f1e4d9a92f486e9b0f59e44
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If one or more cached packs fully covers the request, don't bother
with looking up the objects and trying to walk the graph. Just use
the cached packs and return immediately.
This helps clones of quiet repositories that have not been modified
since their last repack, its likely the cached packs are accurate
and no graph walking is required.
Change-Id: I9062a5ac2f71b525322590209664a84051fd5f8a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
CGit just learned to always use OFS_DELTA when writing out bundle
files. This makes sense because bundle came about well after
OFS_DELTA was established, so any version of CGit that can read a
bundle file can also read OFS_DELTA. Since OFS_DELTA is smaller,
always use it when writing bundles.
Change-Id: I44f9921494798ea0c99e16eab58b87bebeb9aff5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
RevWalk in JGit and the revision code in C Git both parse commits out
of the pack file in an order that differs from strict timestamp and
topological sorting. Both implementations pop a commit from the head
of a date queue, and then immediately parse all of its parents in
order to insert those into the date queue at the proper positions as
determined by their committer timestamp field. This implies that the
parents are parsed when their most recent child is popped from the
queue, and not where they are popped during traversal.
Hoisting a parent commit to be immediately behind its child improves
locality by making sure all parents of a merge are clustered together,
and thus can be paged into the parser by the pack file buffering
system (aka WindowCache in JGit) together.
Change-Id: I80f9e64cafa2e8f082776b43845edf23065386a2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Previously, this method would not (always) work when a recursive path
such as "a/b" was passed into it.
Change-Id: I0752a1f5fc7fef32064d8f921b33187c0bdc7227
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
A checked Exception is thrown instead.
The reason for throwing an Exception is that the state of the
repository is inconsistent in this case: There is a merge
configuration containing a non-existing local branch. Ideally the
deletion of a local branch should also delete the corresponding
merge configuration.
Bug: 337315
Change-Id: I71e56ffb90e11e6e3c1bbd964ad63972d67990c0
Signed-off-by: Stefan Lay <stefan.lay@sap.com>
As PackParser supports a progress meter for the "Resolving deltas"
phase of its work, we should export this to smart HTTP clients so
they know the server is still working on their (large) upload.
However this isn't as simple as just dropping in a binding for
the SmartOutputStream to flush when its told to. We want to
avoid spurious flushes triggered by the use of sideband, or the
status report formatting in the send-pack/receive-pack protocol.
Change-Id: Ibd88022a298c5fed0edb23dfaf2e90278807ba8b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some clients coming through proxies may advertise a different
Accept-Encoding, for example "Accept-Encoding: gzip(proxy)".
Matching by substring causes us to identify this as a false positive;
that the client understands gzip encoding and will inflate the
response before reading it.
In this particular case however it doesn't. Its the reverse proxy
server in front of JGit letting us know the proxy<->JGit link can
be gzip compressed, while the client<->proxy part of the link is not:
client <-- no gzip --> proxy <-- gzip --> JGit
Use a more standard method of parsing by splitting the value into
tokens, and only using gzip if one of the tokens is exactly the
string "gzip". Add a unit test to make sure this isn't broken in
the future.
Change-Id: I30cda8a6d11ad235b56457adf54a2d27095d964e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We also have org.eclipse.jgit.http--All-Tests, which matches the
style of the org.eclipse.jgit.core--All-Tests name. Drop the others
as these are just redundant duplicates.
Change-Id: I8600a343f6a85d21dc07bda68a8cb834c82946b5
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If a cached pack is used, it might know how many deltas are contained
within it. Record that count as part of our reusedDeltas field
for the stats line we show clients.
Change-Id: I1c61fb817305a95eeac654cccf132cba20b2339c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Like ReceivePack, callers that embed UploadPack within their
service may wish to see the set of references that were sent
to the client. We already have the map on hand, it just needs
to be exposed with a getter.
Change-Id: I123b23e475860d5bb968906bef59068985088b7b
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The setMustExist() method allows callers to require the repository
exists in order for build() to succeed. This is useful within a
RepositoryResolver where existence is required.
Change-Id: I6a1154551435cf0da6c2b4a7f4dce266abea5dff
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
DHT based repository types don't use a java.io.File to name the
repository. Moving the type to a string starts to open up more types
of repository names, making the standard pgm package easier to reuse
on other storage systems.
Change-Id: I262ccc8c01cd6db88f832ef317b0e1e5db2d016a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Using a resolver and factory pattern for the anonymous git:// Daemon
class makes transport.Daemon more useful on non-file storage systems,
or in embedded applications where the caller wants more precise
control over the work tasks constructed within the daemon.
Rather than defining new interfaces, move the existing HTTP ones
into transport.resolver and make them generic on the connection
handle type. For HTTP, continue to use HttpServletRequest, and
for transport.Daemon use DaemonClient.
To remain compatible with transport.Daemon, FileResolver needs to
learn how to use multiple base directories, and how to export any
Repository instance at a fixed name.
Change-Id: I1efa6b2bd7c6567e983fbbf346947238ea2e847e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce
Robin Rosenberg
Chris Aniszczyk
Matthias Sohn
Sasa Zivkov
Mathias Kinzler
Stefan Lay
Marc Strapetz
Tomasz Zarna
Jesse Greenwald