The current IgnoreRule/FileNameMatcher implementation scales not well
with huge repositories - it is both slow and memory expensive while
parsing glob expressions (bug 440732). Addtitionally, the "double star"
pattern (/**/) is not understood by the old parser (bug 416348).
The proposed implementation is a complete clean room rewrite of the
gitignore parser, aiming to add missing double star pattern support and
improve the performance and memory consumption.
The glob expressions from .gitignore rules are converted to Java regular
expressions (java.util.regex.Pattern). java.util.regex.Pattern code can
evaluate expression from gitignore rules considerable faster (and with
less memory consumption) as the old FileNameMatcher implementation.
CQ: 8828
Bug: 416348
Bug: 440732
Change-Id: Ibefb930381f2f16eddb9947e592752f8ae2b76e1
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Applications sometimes use a RevFlag instead of a Set<RevObject>
to track boolean state bits about objects being processed. However
this requires careful use of the resetRetain() methods to avoid an
accidental clearing of the RevFlag bits, effectively clearing the
Set<RevObject> the application wanted to track.
Simplify that use case by offering retainOnReset, a collection of
flags that are never cleared by the RevWalk.
Change-Id: I4c05b89b1398e4a4f371eac3a5d1d5edddec838f
ResetCommand threw an NPE if neither mode nor path was defined. Instead
it should default to a mixed reset like native git does.
Change-Id: I455902394f9e7b0c7afae42381f34838f7f2a138
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When during an ObjectWalk commits are marked as uninteresting we should
be tolerant against the situation that the commit exists in the repo but
the referenced tree is not exisiting. Since commit
c4797fe986 we are throwing
MissingObjectException in such a case. This semantic differs from native
git behaviour and may cause push operations to fail while they would
work in native git. See:
http://dev.eclipse.org/mhonarc/lists/egit-dev/msg03585.html
Bug: 445744
Change-Id: Ib7dec10fd2ef1adbb8adbabb9d3d5a64e554286a
When during an ObjectWalk commits are marked as uninteresting we should
be tolerant against the situation that the commit exists in the repo but
the referenced tree is not exisiting. Since commit
c4797fe986 we are throwing
MissingObjectException in such a case. This semantic differs from native
git behaviour and may cause push operations to fail while they would
work in native git. See:
http://dev.eclipse.org/mhonarc/lists/egit-dev/msg03585.html
Bug: 445744
Change-Id: Ib7dec10fd2ef1adbb8adbabb9d3d5a64e554286a
When writing new packs it should be allowed to specify objects as "have"
(objects which should not be included in the pack) which do not exist in
the local repository.
This works with the traditional PackWriter, but when PackWriter was
working on a repository with bitmap indexes and used
PackWriterBitmapWalker then this feature was broken. Non-existing "have"
objects lead to MissingObjectExceptions. That broke push and Gerrit
replication. When the replication target had branches unknown to the
replication source then the source repository wanted to build pack files
where "have" included branch-tips which were unknown in the source
repository.
Bug: 427107
Change-Id: I6b6598a1ec49af68aa77ea6f1f06e827982ea4ac
Also-by: Matthias Sohn <matthias.sohn@sap.com>
The chaining credentials provider sequentially tries to obtain
credentials from a list of credential providers and returns the
credentials from the first provider which can provide them.
Change-Id: I499f304119d7066d011dbde3556dee6facee8ab0
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
JGit should offer the possibility to do a garbage collection in
"aggressive" mode. In this mode garbage collection more aggressively
optimize the repository at the expense of taking much more time.
Technically a aggressive mode garbage collection differs from a
non-aggressive one by:
- not reusing packed objects found in old packs. Recompress every object
- the configuration pack.window is set to 250 (the default is 10)
- the configuration pack.depths is set to 250 (the default is 50)
The associated classes in org.eclipse.jgit.api and the command line
command in org.eclipse.jgit.pgm expose this new option.
The configuration parameters gc.aggressiveDepth and gc.aggressiveWindow
have been introduced to configure this feature.
Bug: 444332
Change-Id: I024101f2810acf6be13ce144c9893d98f5c4ae76
Do not silently set the NetRCCredentialsProvider if no
CredentialsProvider was set explicitly since applications may want to
have full control which provider should be used.
Bug: 444338
Change-Id: Ie096983bc1caa90443a504d302bfea8f2d26ab9e
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It can be useful for a server operator to know how long a pack
writer spent writing out objects, even if the request aborts and
never finishes.
Record more of the stats data inside of a finally block, to
ensure these can be included into the server's monitoring.
Change-Id: I00858aa393a948f8e742e64ae4c00953eadaef95
Some of these eliminations just reduces the number of warnings on
lines where messages are constructed that can/will be translated.
Change-Id: I6eddb39ccc8f2488741bb58540d9ec5f5665e2c4
Other .git files are not hidden with this patch
Change-Id: Idf63ca08d08f3a77c33f5848d02074f8d6a75758
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If the git server requires authentication and no CredentialsProvider is
registered TransportHttp.connect() would throw an NPE since it tries to
reset the credentials provider. Instead throw a TransportException
explaining the problem.
Change-Id: Ib274e7d9c43bba301089975423de6a05ca5169f6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Set the commit message to be used for the merge commit (in case one is
created)
Bug: 442886
Change-Id: Ie5ecc13822faa366f00b3daa07f74c8441cae195
Signed-off-by: Axel Richard <axel.richard@obeo.fr>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If the packer fails, still obtain the stats and make them available
to the logger and the caller. Failures can frequently happen when
a client disconnects in the middle of a pack stream. Server admins
may still want to examine the timing metrics from counting and
compressing phases.
Change-Id: Iceae4f68b5473f4223d85c9edfb57837fc818eed
In such case, we use the name attribute as the default value of path.
Change-Id: I53fa312d6b64c6eb2240f08af7d1d60cea99192a
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
uninteresting
Using the ObjectWalk and marking a commit as uninteresting didn't mark
its root tree as uninteresting. This caused the "missing tree ..."
error in Gerrit under special circumstances. For example, if the
patch-set 2 changes only the commit message then the patch-set 1
and patch-set 2 share the same root-tree:
ps1 -> o o <- ps2
\ /
o root-tree
The transported pack will contain the ps2 commit but not the root-tree
object.
When using the BaseReceivePack.setCheckReferencedObjectsAreReachable
JGit will check the reachability of all referenced objects not provided
in the transported pack. Since the ps1 was advertised it will properly
be marked as uninteresting. However, the root-tree was reachable because
the ObjectWalk.markUninteresting missed to mark it as uninteresting.
JGit was then rejecting the pack with the "missing tree ..." exception.
Gerrit-issue: https://code.google.com/p/gerrit/issues/detail?id=1582
Change-Id: Iff2de8810f14ca304e6655fc8debeb8f3e20712b
Signed-off-by: Saša Živkov <sasa.zivkov@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
In Git 1.9 (5dbd767601 "support pushing from a shallow clone")
the git-core project intentionally broke the existing send-pack
protocol from shallow clients.
Shallow clients now transmit their shallow information during push,
ahead of the old-new command sequence. JGit must accept these lines
when presented.
To protect the server against clients sending partial history,
require the connectivity check when pushed to by a shallow client.
Change-Id: I46639366b0900052c376091e1688f07def44ab79
This matches what C Git does, see "stripped" in `man git-commit-tree`.
It also fixes the bug of the user where an user.email like "<>" would
show up as "<<>>" in EGit.
Bug: 439844
Change-Id: I567a3c620e191ce9d37d318417e63cb5d4483419
Signed-off-by: Robin Stocker <robin@nibor.org>
URIish.unescape() threw an ArrayIndexOutOfBoundsException if the given
url has incorrect percent-encoding (e.g. http://example.com/%gg). But an
URISyntaxException is much better to know the reason of the failure.
Change-Id: I3f40a26d43cd2eb4e32c11aba7dc2594bc1f98e2
Signed-off-by: Yi EungJun <eungjun.yi@navercorp.com>
This is necessary to ensure objects accessed by the TreeWalk come from
the associated ObjectInserter when the merger is a RecursiveMerger
instance and a virtual common base was constructed but not flushed.
Change-Id: Iebe739d30fd868ebc4f61dbfb714673146a2c3ec
According to http://stackoverflow.com/a/8381338, the maximum array
size is not Integer.MAX_VALUE, but Integer.MAX_VALUE - 8
Change-Id: I6ddc7470368acd20abf0885c53c89a982bb0f176
Signed-off-by: Marc Strapetz <marc.strapetz@syntevo.com>
The native implementation of inflate() can set finished to return
true at the same time as it copies the last bytes into the buffer.
Check for finished on each iteration, terminating as soon as libz
knows the stream was completely inflated.
If not finished, it is likely input is required before the next
native call could do any useful work. Most invocations are passing
in a buffer large enough to store the entire result. A partial return
from inflate() will need more input before it can continue. Checking
right away that needsInput() is true saves a native call to determine
no bytes can be inflated without more input.
This should fix a rare infinite loop condition inside of inflation
when an object ends exactly at the end of a block boundary, and
the next block contains only the 20 byte trailing SHA-1.
When the stream is finished each new attempt to inflate() returns
n == 0, as no additional bytes were output. The needsInput() test
tries to add the length of the footer block to itself, but then loops
back around an reloads the same block as the block is smaller than
a full block size. A zero length input is set to the inflater,
which triggers needsInput() condition again.
Change-Id: I95d02bfeab4bf995a254d49166b4ae62d1f21346
Andrey Loskutov
Shawn Pearce
Robin Stocker
Matthias Sohn
Christian Halstrick
Marc Strapetz
Robin Rosenberg
Shawn Pearce
Konrad Kügler
Axel Richard
Yuxuan 'fishy' Wang
Saša Živkov
Yi EungJun