DfsGarbageCollector asks PackWriter for the set of objects packed
after the bitmap index is written out. This is now null as it was
cleared to release memory. Instead use PackBitmapIndexBuilder to
build the set as it also has the objects.
Reduce memory in PackBitmapIndexBuilder by fully discarding the
ObjectToPack instances. This was the original intent of commit
4bb523475d ("PackWriter: shed memory while creating bitmaps")
but failed as the instances were still held live here.
Switch to BlockList instead of ObjectToPack[]. This allows the
JVM to allocate many smaller arrays instead of one contiguous
array with 5.2M reference pointers. In a tight heap the smaller
allocations are more feasible.
Reduce the initial EWAHCompressedBitmaps for the 4 type maps. On
average a typical repository is 30% commits, 30% trees and 30% blobs.
These bitmaps are typically very dense. PackWriter orders objects by
commit, tree, blob when writing the file so these should always be a
very dense run of 1s with some 0s before and after. So even the 1/3rd
allocation is likely too large, but the later trim() will reduce the
internal buffer anyway.
Change-Id: If0b80a31cb00894f1485ff8f53ef7ae5a759a046
After jgit moved to Java 7 there is no need in an extra
Java7BasicAttributes class. Also all fields of Attributes can be made
final now.
Change-Id: I0be6daf7758189b0eecc4e26294bd278ed8bf7a0
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Once bitmap creation begins the internal maps required for packing are
no longer necessary. On a repository with 5.2M objects this can save
more than 438 MiB of memory by allowing the ObjectToPack instances to
get garbage collected away.
Downside is the PackWriter cannot be used for any further opertions
except to write the bitmap index. This is an acceptable trade-off as
in practice nobody uses the PackWriter after the bitmaps are built.
Change-Id: Ibfaf84b22fa0590896a398ff659a91fcf03d7128
For construction performance each new EWAHBitmap is allocated at the
roughly worst-case size the bitmap would need if all of the words must
be literal and no run length compression is available. In practice
this is far larger than is required, wasting heap memory while the
bitmaps are computed.
Trim down each bitmap to its minimum required size. This copies the
internal array to a new smaller array, allowing the GC to reclaim the
prior larger array for reuse.
A single bitmap of 5.2M bits is only 79 KiB of memory without this
trim call but 15,000 such bitmaps is 1.1 GiB. Trimming can help fit
a larger number of bitmaps during processing.
Change-Id: I2bd19a786189db5b01c4c96f209b83de50e10c3b
The bitmap preparer only needs commit graph topology; it does not use
the message body. Allow the RevWalk to free the body after the commit
has been parsed to save memory.
Change-Id: I97d4a440c9fc313873fd224bd05b9d9e3dc575db
The WorkingTreeIterator.isEntryIgnored() should use originally requested
file mode while descending to the file tree root and checking ignore
rules. Original code asking isEntryIgnored() on a file was using
directory mode instead if the .gitignore was not located in the same
directory.
Bug: 473506
Change-Id: I9f16ba714c3ea9e6585e9c11623270dbdf4fb1df
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
In addition to honor the http_proxy variable for setting a proxy for
http JGit should also honor the https_proxy variable to set a similar
proxy for https traffic
Bug: 473365
Change-Id: I1002cb575e26cd842bf81ad751ec7c267b585ce2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When during Merge for a certain path OURS & BASE contains a file and
THEIRS contains a folder there was a bug in JGit leading to unnecessary
conflicts. This commit fixes it and adds a test for this situation.
Bug: 472693
Change-Id: I71fac5a6a2ef926c01adc266c6f9b3275e870129
Also-by: Clemens Buchacher <drizzd@aon.at>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
According to [1] leading spaces are allowed in ignore rules and trailing
spaces are allowed too if they are escaped via backslash.
[1] https://www.kernel.org/pub/software/scm/git/docs/gitignore.html
Bug: 472762
Change-Id: I5e3ae5599cb9e5d80072f38c82c20cbc9475a18a
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Catch unexpected PatternSyntaxException and convert it to
InvalidPatternException. Log such errors, do not silently ignore them.
Bug: 463581
Change-Id: Id0936d9816769ec0cfae1898beda0f7a3c146e67
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
According to [1] backslash can escape leading special characters '#' and
'!' in ignore rules, so that they are treated literally.
[1] https://www.kernel.org/pub/software/scm/git/docs/gitignore.html
Bug: 463581
Change-Id: I4c02927413a9c63ea5dbf2954877080d902ec1b2
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
While checking if we should consider an ignore rule without '[]'
brackets as a regular expression, check if the backslash escapes one of
the glob special characters '?', '*', '[', '\\'. If not, backslash is
not a part of a regex and should be treated literally.
Bug: 463581
Change-Id: I85208c7f85246fbf6c5029ce3c8b7bb8f4dbd947
Signed-off-by: Andrey Loskutov <loskutov@gmx.de>
Rather than lazily parsing the push in this method, parse it at the
end of recvCommands(), which already contains the necessary try/catch
for handling this error. This allows later callers to avoid having to
handle this condition superfluously.
Change-Id: I5dcaf1a44bf4e321adc281e3381e7e17ac89db06
Consider a BatchRefUpdate produced by Gerrit Code Review, where the
original command pushed over the wire might refer to
"refs/for/master", but that command is ignored and replaced with some
additional commands like creating "refs/changes/34/1234/1". We do not
want to store the cert in "refs/for/master@{cert}", since that may
lead someone looking to the ref to the incorrect conclusion that that
ref exists.
Add a separate put method that takes a collection of commands, and
only stores certs on those refs that have a matching command in the
cert.
Change-Id: I4661bfe2ead28a2883b33a4e3dfe579b3157d68a
- update org.apache.httpcomponents.httpcore to 4.3.3
- update org.apache.httpcomponents.httpclient to 4.3.6, 4.3.5 and later
are reported to fix vulnerability CVE-2014-3577
CQ: 9220
CQ: 9221
Bug: 470523
Change-Id: I39b80b250c6c1daede6a23e9f177fb2988ac37bb
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
37a1e4be moved this constant causing the following error message in
Eclipse: "The static field LocalDiskRepositoryTestCase.CONTENT should be
accessed directly".
Change-Id: I4ceb57a30f2e5a8f7e55109ef260a244ed5e7044
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Inspired by a proposal from gitolite[1], where we store a file in
a tree for each ref name, and the contents of the file is the latest
push cert to affect that ref.
The main modification from that proposal (other than lacking the
out-of-git batching) is to append "@{cert}" to filenames, which allows
storing certificates for both refs/foo and refs/foo/bar. Those
refnames cannot coexist at the same time in a repository, but we do
not want to discard the push certificate responsible for deleting the
ref, which we would have to do if refs/foo in the push cert tree
changed from a tree to a blob.
The "@{cert}" syntax is at least somewhat consistent with
gitrevisions(7) wherein @{...} describe operators on ref names.
As we cannot (currently) atomically update the push cert ref with the
refs that were updated, this operation is inherently racy. Kick the can
down the road by pushing this burden on callers.
[1] cf062b8bb6/contrib/hooks/repo-specific/save-push-signatures
Change-Id: Id3eb32416f969fba4b5e4d9c4b47053c564b0ccd
This will allow us to write the super project in a branch other than
master.
Change-Id: I578ed9ecbc6423416239e31ad644531dae9fb5c3
Signed-off-by: Yuxuan 'fishy' Wang <fishywang@google.com>
The test helper method `indexState` in `RepositoryTestCase` is
very useful for writing tests, even in cases where we need to
do things like create more than one repository for a test and
thus we don't want to use the built-in `db` member variable that
exists in `RepositoryTestCase`. Since the method is static,
we can move it up to the parent class `LocalDiskRepositoryTestCase`,
where it can be used by tests that aren't a great fit for inheriting
directly from `RepositoryTestCase`.
Bug: 436200
Change-Id: I2b6de75c001d2d77ddb607488af246548784a67f
Signed-off-by: Chris Price <chris@puppetlabs.com>
The `indexState` method is no longer referring to any
member variables from the class, so it can be made static.
Bug: 436200
Change-Id: I013316de5c373417ea758ca6e17da29209fead53
Signed-off-by: Chris Price <chris@puppetlabs.com>
Primarily to aid in writing end-to-end tests, which must be written
outside this package as we don't currently have a Bouncy Castle
dependency.
Change-Id: Ia24179773140d9ecc616b3a6b2e630c85d800157
We have done this since forever with the "wanted old new ref" error,
so let's do it for other such errors thrown in the same block as well.
Change-Id: Ib3b1c7f05e31a5b3e40e85eb07b16736920a033b
When pushing to an HTTP server using the C git client, I observed a
certificate lacking a pushee field. Handle this gracefully in the
parser.
Change-Id: I7f3c5fa78f2e35172a93180036e679687415cac4
Shawn Pearce
Andrey Loskutov
Christian Halstrick
Dave Borowitz
Matthias Sohn
Terry Parker
Jonathan Nieder
Yuxuan 'fishy' Wang
Chris Price