Implements a garbage collector for FileRepositories. Main ideas are
copied from the garbage collector for DFS based repos
(DfsGarbageCollector). Added functionalities are
- pruning loose objects
- handling of the index
- packing refs
- handling of reflogs (objects referenced from reflog will not be
pruned/)
These are features of a GC which are not handled in this change and
which should come with subsequent changes:
- unpacking packed objects into loose objects (to support that pruning
packed objects doesn't delete them until they are older than two weeks)
- expiration of reflogs
- support for configuration parameters (e.g. gc.pruneExpire)
Change-Id: I14ea5cb7e0fd1b5c50b994fd77f4e05bfbb9d911
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Expose class DfsReader and method DfsPackFile.hasObject() as public.
Applications may want to be able to inquire about some details of
the storage of a repository. Make this possible by exposing some
simple accessor methods.
Expose method DfsObjDatabase.clearCache() as protected, allowing
implementing subclasses to dump the cache if necessary, and force
it to reload on a future request.
Change-Id: Ic592c82d45ace9f2fa5f8d7e4bacfdce96dea969
Currently, after a merge/cherry-pick/rebase, all index entries are
smudged as the ResolveMerger never sets entry lengths and/or
modification times. This change teaches it to re-set them at least for
things it did not touch. The other entries are then repaired when the
index is persisted, or entries are checked out.
The first attempt to get this in was commit
3ea694c252 which has been reverted.
Since then some fixes to ResolveMerger and a few more tests have
been added which check situations where the index is not matching
HEAD before we merge.
Change-Id: I648fda30846615b3bf688c34274c6cf4bc857832
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Also-by: Markus Duft <markus.duft@salomon.at>
Currently, after a merge/cherry-pick/rebase, all index entries are
smudged as the ResolveMerger never sets entry lengths and/or
modification times. This change teaches it to re-set them at least for
things it did not touch. The other entries are then repaired when the
index is persisted, or entries are checked out.
Change-Id: I0944f2017483d32043d0d09409b13055b5609a4b
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Currently, update manager displays the comment message (starting with
"###...") which should not be part of the description.
Change-Id: I1c748cd5987b30f25aca3ca8eb593448bcf128ae
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Otherwise applying will fail with a FileNotFoundException, because
File.createNewFile() fails with missing parents.
Contains change & according test.
Change-Id: I970522b549b8bb260ca6720da11f12c57ee8a492
Signed-off-by: Chris Aniszczyk <zx@twitter.com>
It's used in DhtMissingChunkException, which is serializable.
Change-Id: I2b76bc1bc373efd44214be4598a03c62c681a200
Signed-off-by: Robin Stocker <robin@nibor.org>
An example where this is necessary is when a whole directory was deleted
and checkout is used to restore a file which was in that directory.
Bug: 372133
Change-Id: I1d45e0a5d2525fe1fdfbf08c9c5c166dd909e9fd
Signed-off-by: Robin Stocker <robin@nibor.org>
* stable-2.0:
Improve performance of persisting an index by magnitudes
Change-Id: I880f4d0be81c263b598ee24e8373c6f8cb781c51
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When updating smudged entries use a pathfilter to iterate only over
working tree files which have an associated smudged index entry.
Commit dac66672df introduced that we
check and update smudged entries while persisting a dircache. Before
that commit adding a file to git caused file i/o for the index file,
the object database (to store new content) and all files we wanted to
add (to read new content). After that commit we have additionally file
i/o for every file in the working tree (even ignored files).
Especially on windows iterating over the working tree can be very
time consuming. This decreased the performance of persisting a dircache
dramatically (I measured factors 4 to 10 while adding a file to linux
kernel repo). One could easily see this effect when dragging&dropping
modified files in a linux kernel repo in the egit staging view.
Change-Id: I568dec77635013cf6bb46f652d3f2b89de041c82
Signed-off-by: Kevin Sawicki <kevin@github.com>
I have unfortunately introduced a few bugs in the native Git client
over the years. 1.7.5 is unable to send chunked requests correctly,
resulting in corrupt data at the server. Ban this client whenever
it uses chunked encoding with an error message.
Prior to some more recent versions, git push over HTTP failed to
report status information and error messages due to a race within
the client and its helper process. Check for these bad versions and
send errors as messages before the status report, enabling users
to see the failures on their terminal.
Change-Id: Ic62d6591cbd851d21dbb3e9b023d655eaecb0624
It was easy to create multiple packs with exactly the same name and
same DfsRepositoryDescription in a test, which can poison the
DfsBlockCache. The javadoc for DfsObjDatabase.newPack() explicitly
says pack names should be unique within an entire DFS, so do this by
making the packId AtomicInteger static.
Arguably, test writers shouldn't be doing things like putting
'new DfsRepositoryDescription("test")' in a setUp() method, but that's
a natural thing to do, and we don't document this restriction
anywhere.
Change-Id: I9477413ab3950d83b7d17e173fbc0a3e064896e3
The PackParser EOF check is incompatible with the expect data after
pack footer flag, so turn off the EOF check if the expecting data
flag is true.
Change-Id: I697ebd9e1d1eed765d00aecaef955cf978cfd0b9
This was broken in fe1f1b8f8a, which
preferred the index over the working tree when both were present.
Change-Id: I97dcf9a088adcbd0187fa7eec9ef34445ce3a981
Signed-off-by: Kevin Sawicki <kevin@github.com>
When starting a rebase with C Git, there may be empty lines in the
git-rebase-todo file. Before this change, JGit would fail to parse the
file with e.g. the following exception:
JGitInternalException: Unknown or unsupported command "
#", only "pick" is allowed.
This happened when there was an empty line just before the comments,
because the nextSpace would be the one from the comment. Now the empty
lines are ignored by checking for nextSpace < ptr outside of the loop.
Change-Id: I94ad299f367c846e7729c74f49c6b8f93f75ae81
Signed-off-by: Robin Stocker <robin@nibor.org>
Before, the paths to delete were stored in a HashMap, which doesn't have
a particular order. So when e.g. both the file "a/b" and the directory
"a" were to be deleted, it would sometimes try to delete "a" first. This
resulted in a failed path because File#delete() fails when a directory
isn't empty.
With this change, an ArrayList is used for storing the paths to delete.
The list contains the paths in a top-down order, as defined by the order
of processEntry. When the files are deleted, the list is iterated in
reverse, ensuring that all files of a directory are deleted before the
directory itself.
Bug: 354099
Change-Id: I6b2ce96b3932ca84ecdfbeab457ce823c95433fb
Signed-off-by: Robin Stocker <robin@nibor.org>
ResolveMerger#mergeImpl() was only returning false (= failed) when there
were unmerged paths. In the case when there were only failing paths, it
returned true.
Because MergeCommand looks at the return value for determining if the
merge failed, it would fall into the successful case there, where it
should instead return a MergeResult with MergeStatus.FAILED.
This change adds a test case for this and makes the ResolveMerger return
false when there are failing paths.
This was discovered while working on fixing bug 354099 and is needed for
its test case.
Bug: 354099
Change-Id: I499f518f6289ef93e017db924b2aa857f2154707
Signed-off-by: Robin Stocker <robin@nibor.org>
EGit needs this to be able to revert unsaved changes.
Change-Id: I50cc8056aaff47fef6080970866962e3eb634e29
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
This has no effect on Git clients, but for browsers, 403 Forbidden may
be more appropriate. 500 Internal Server Error implies that there is
a problem with the server, whereas ServiceMayNotContinueException is
specifically intended to cover cases where the server is functioning
correctly but has determined that the request may not proceed.
Change-Id: I825abd2a029d372060103655eabf488a0547c1e8
Whenever a call to JGit returns a Repository the caller should make sure
to call close() on it if he doesn't need it anymore. Since instances of
Repository contain e.g. open FileOutputStreams (for pack files)
forgetting to close the repository can lead to resource leaks.
This was the reason why dozens of the JUnit tests failed on Windows
with "Can't delete file ...." errors.
In LocalDiskRepositoryTestCase.tearDown() we tried to delete the
repositories we used during tests which failed because we had open
FileOutputStreams.
Not only the obvious cases during Clone or Init operations returned
Repositories, but also the new SubModule API created repository
instances. In some places we even forgot to close submodule repositories
in our internal coding.
To see the effects of this fix run the JGit JUnit tests under Windows.
On other platforms it's harder to see because either the leaking
resources don't lead to failing JUnit tests (on Unix you can delete
files with open FileOutputStreams) or the java gc runs differently and
cleans up the resources earlier.
Change-Id: I6d4f637b0d4af20ff4d501db091548696373a58a
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When receiving a pack, data buffered after the pack can restored
to the InputStream if the stream supports mark and reset.
Change-Id: If04915c32c91be28db8df7e8491ed3e9fe0e1608
LockFileTest was failing on Windows because we couldn't delete the lock
file of the index. The reason was that a LockFile instance still had an
open handle to the lock file preventing us to delete the file (in
contrast to the behavior on other platforms).
Change-Id: I1d50442b7eb8a27f98f69ad77c5e24a9698a7b66
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Split Service into MultiRequestService (fetch, push) and
LongPollService (upcoming publish-subscribe).
Change-Id: Ice373d3dee63c395490d2707473ccf20a022e5cf
It is not always appropriate to use the .gitmodules file from the
working tree, for example if reading the modules at a specific commit.
And sometimes it is impossible, as in a bare repository.
When using the static factory methods, automatically set up the
appropriate root tree so lazy loading of the config file reads from
the appropriate place. Leave the current behavior of looking in the
working tree as a fallback for the case where walking the index.
Change-Id: I71b7ed3ba16c80b0adb8c5fd85b5c37fd4aef8eb
We need a way to release the underlying TreeWalk. Also, use this
method to release walks from the static factory methods on error or
when submodules are not found.
Change-Id: I6bedc2db78bcd577aef2cfe6715bb981a26dcfd7
Robin Rosenberg
Christian Halstrick
Shawn O. Pearce
Shawn Pearce
Markus Duft
Tomasz Zarna
Robin Stocker
Matthias Sohn
Dave Borowitz
Ian Wetherbee