By using RefUpdate for symbolic reference creation we can reuse
the logic related to updating the reflog with the event, without
needing to expose something such as the legacy ReflogWriter class
(which we no longer have).
Applications using writeSymref must update their code to use the
new pattern of changing the reference through the updateRef method:
String refName = "refs/heads/master";
RefUpdate u = repository.updateRef(Constants.HEAD);
u.setRefLogMessage("checkout: moving to " + refName, false);
switch (u.link(refName)) {
case NEW:
case FORCED:
case NO_CHANGE:
// A successful update of the reference
break;
default:
// Handle the failure, e.g. for older behavior
throw new IOException(u.getResult());
}
Change-Id: I1093e1ec2970147978a786cfdd0a75d0aebf8010
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This commit actually does three major changes to the way references
are handled within JGit. Unfortunately they were easier to do as
a single massive commit than to break them up into smaller units.
Disambiguate symbolic references:
---------------------------------
Reporting a symbolic reference such as HEAD as though it were
any other normal reference like refs/heads/master causes subtle
programming errors. We have been bitten by this error on several
occasions, as have some downstream applications written by myself.
Instead of reporting HEAD as a reference whose name differs from
its "original name", report it as an actual SymbolicRef object
that the application can test the type and examine the target of.
With this change, Ref is now an abstract type with different
subclasses for the different types.
In the classical example of "HEAD" being a symbolic reference to
branch "refs/heads/master", the Repository.getAllRefs() method
will now return:
Map<String, Ref> all = repository.getAllRefs();
SymbolicRef HEAD = (SymbolicRef) all.get("HEAD");
ObjectIdRef master = (ObjectIdRef) all.get("refs/heads/master");
assertSame(master, HEAD.getTarget());
assertSame(master.getObjectId(), HEAD.getObjectId());
assertEquals("HEAD", HEAD.getName());
assertEquals("refs/heads/master", master.getName());
A nice side-effect of this change is the storage type of the
symbolic reference is no longer ambiguous with the storge type
of the underlying reference it targets. In the above example,
if master was only available in the packed-refs file, then the
following is also true:
assertSame(Ref.Storage.LOOSE, HEAD.getStorage());
assertSame(Ref.Storage.PACKED, master.getStorage());
(Prior to this change we returned the ambiguous storage of
LOOSE_PACKED for HEAD, which was confusing since it wasn't
actually true on disk).
Another nice side-effect of this change is all intermediate
symbolic references are preserved, and are therefore visible
to the application when they walk the target chain. We can
now correctly inspect chains of symbolic references.
As a result of this change the Ref.getOrigName() method has been
removed from the API. Applications should identify a symbolic
reference by testing for isSymbolic() and not by using an arcane
string comparsion between properties.
Abstract the RefDatabase storage:
---------------------------------
RefDatabase is now abstract, similar to ObjectDatabase, and a
new concrete implementation called RefDirectory is used for the
traditional on-disk storage layout. In the future we plan to
support additional implementations, such as a pure in-memory
RefDatabase for unit testing purposes.
Optimize RefDirectory:
----------------------
The implementation of the in-memory reference cache, reading, and
update routines has been completely rewritten. Much of the code
was heavily borrowed or cribbed from the prior implementation,
so copyright notices have been left intact as much as possible.
The RefDirectory cache no longer confuses symbolic references
with normal references. This permits the cache to resolve the
value of a symbolic reference as late as possible, ensuring it
is always current, without needing to maintain reverse pointers.
The cache is now 2 sorted RefLists, rather than 3 HashMaps.
Using sorted lists allows the implementation to reduce the
in-memory footprint when storing many refs. Using specialized
types for the elements allows the code to avoid additional map
lookups for auxiliary stat information.
To improve scan time during getRefs(), the lists are returned via
a copy-on-write contract. Most callers of getRefs() do not modify
the returned collections, so the copy-on-write semantics improves
access on repositories with a large number of packed references.
Iterator traversals of the returned Map<String,Ref> are performed
using a simple merge-join of the two cache lists, ensuring we can
perform the entire traversal in linear time as a function of the
number of references: O(PackedRefs + LooseRefs).
Scans of the loose reference space to update the cache run in
O(LooseRefs log LooseRefs) time, as the directory contents
are sorted before being merged against the in-memory cache.
Since the majority of stable references are kept packed, there
typically are only a handful of reference names to be sorted,
so the sorting cost should not be very high.
Locking is reduced during getRefs() by taking advantage of the
copy-on-write semantics of the improved cache data structure.
This permits concurrent readers to pull back references without
blocking each other. If there is contention updating the cache
during a scan, one or more updates are simply skipped and will
get picked up again in a future scan.
Writing to the $GIT_DIR/packed-refs during reference delete is
now fully atomic. The file is locked, reparsed fresh, and written
back out if a change is necessary. This avoids all race conditions
with concurrent external updates of the packed-refs file.
The RefLogWriter class has been fully folded into RefDirectory
and is therefore deleted. Maintaining the reference's log is
the responsiblity of the database implementation, and not all
implementations will use java.io for access.
Future work still remains to be done to abstract the ReflogReader
class away from local disk IO.
Change-Id: I26b9287c45a4b2d2be35ba2849daa316f5eec85d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These types can be used by RefDatabase implementations to manage
the collection.
A RefList stores items sorted by their name, and is an immutable
type using copy-on-write semantics to perform modifications to
the collection. Binary search is used to locate an existing item
by name, or to locate the proper insertion position if an item does
not exist.
A RefMap can merge up to 3 RefList collections at once during its
entry iteration, allowing items in the resolved or loose RefList
to override items by the same name in the packed RefList.
The RefMap's goal is O(log N) lookup time, and O(N) iteration time,
which is suitable for returning from a RefDatabase. By relying on
the immutable RefList we might be able to make map construction
nearly constant, making Repository.getAllRefs() an inexpensive
operation if the caches are current. Since modification is not
common, changes require up to O(N + log N) time to copy the internal
list and collapse or expand the list's array. As most changes
are made to the loose collection and not the packed collection,
in practice most changes would require less than the full O(N)
time, due to a significantly smaller N in the loose list.
Almost complete test coverage is included in the corresponding
unit tests. A handful of methods on RefMap are not tested in this
change, as writing the proper test depends on a future refactoring
of how the Ref class represents symbolic reference names.
Change-Id: Ic2095274000336556f719edd75a5c5dd6dd1d857
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Not all occurrences of ".git" are replaced by this constant, only
those where it actually refers to the directory with that name, i.e
not the ".git" directory suffix.
Asserts and comment are also excluded from replacement.
Change-Id: I65a9da89aedd53817f2ea3eaab4f9c2bed35d7ee
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Later we are going to add support for smart HTTP, which requires us to
buffer at least some of the request created by a client before we ship
it to the server. For many requests, we can fit it completely into a
1 MiB buffer, but if it doesn't we can drop back to using the chunked
transfer encoding to send an unknown stream length.
Rather than recoding the block based memory buffer, we refactor the
local file overflow strategy into a subclass, allowing the HTTP client
code to replace this portion of the logic with its own approach to
start the chunked encoding request.
Change-Id: Iac61ea1017b14e0ad3c4425efc3d75718b71bb8e
Signed-off-by: Shawn O. Pearce <sop@google.com>
The multi_ack_detailed extension breaks out the "ACK %s continue" status
code into "ACK %s common" and "ACK %s ready" states, making it easier to
discover which objects are truely common, and which objects are simply
on a chain the server doesn't care learning about.
Change-Id: Ie8e907424cfbbba84996ca205d49eacf339f9d04
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
These routines create a fairly clean DSL for writing out the
structure of a repository in a test case. Abstract them into
a helper class that we can reuse in other test environments.
Change-Id: I55cce3d557e1a28afe2fdf37b3a5b67e2651c9f1
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Other test suites may find this useful, especially when trying
to defeat the pack file compression with random data files.
Change-Id: Ic00a4ac626af7a1c94d18ee99305e295b267b1a3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since Robin reverted using the maven-bundle-plugin to produce the
OSGi manifest, there is no reason for us to reference it from our
build process anymore.
Also, when Robin reverted the to the Eclipse way of doing things,
we failed to update the ignore files to ignore our generated files
but not ignore our tracked .classpath.
Finally, we cannot delete the MANIFEST.MF file during a Maven build,
as this is once again a source file.
Change-Id: I53f77f2002cb4285f728968829560e835651e188
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This restores the ability to build using just Eclipse without
strange procedures, extra plugins and it is again possible to
work on both JGit and EGit in the same Eclipse workspace with
ease.
Change-Id: I0af08127d507fbce186f428f1cdeff280f0ddcda
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Adds the file content merge alorithm and tests for merge to jgit.
The merge algorithm:
- Gets as input parameters the common base, the two new contents
called "ours" and "theirs".
- Computes the Edits from base to ours and from base to theirs with
the help of MyersDiff.
- Iterates over the edits.
- Independent edits from ours or from theirs will just be applied
to the result.
- For conflicting edits we first harmonize the ranges of the edits
so that in the end we have exactly two edits starting and ending
at the same points in the common base. Then we write the two
conclicting contents into the result stream.
Change-Id: I411862393e7bf416b6f33ca55ec5af608ff4663
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
[sp: Fixed up two awkard comments in documentation.]
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The UnionInputStream utility class combines multiple sequential
InputStreams so they appear to the caller as a single stream with
no gaps. This can be used to concentate streams coming from multiple
independent HTTP connections (for example).
The companion unit test covers the class's full functionality.
Change-Id: I0676c7b5e082a5886bf0e8f43f9fd6c46a666228
Signed-off-by: Shawn O. Pearce <sop@google.com>
Tycho isn't production ready for projects like JGit to be using as
their primary build driver. Some problems we ran into with Tycho
0.6.0 that are preventing us from using it are:
* Tycho can't run offline
The P2 artifact resolver cannot perform its work offline. If the
build system has no network connection, it cannot compile a
project through Tycho. This is insane for a distributed version
control system where developers are used to being offline during
development and local testing.
* Magic state in ~/.m2/repository/.meta/p2-metadata.properties
Earlier iterations of this patch tried to use a hybrid build,
where Tycho was only used for the Eclipse specific feature and P2
update site, and maven-bundle-plugin was used for the other code.
This build seemed to work, but only due to magic Tycho specific
state held in my local home directory. This means builds are not
consistently repeatable across systems, and lead me to believe
I had a valid build, when in fact I did not.
* Manifest-first build produces incomplete POMs
The POM created by the manifest-first build format does not
contain the dependency chain, leading a downstream consumer to
not import the runtime dependencies necessary to execute the
bundle it has imported. In JGit's case, this means JSch isn't
included in our dependency chain.
* Manifest-first build produces POMs unreadable by Maven 2.x
JGit has existing application consumers who are relying on
Maven 2.x builds. Forcing them to step up to an alpha release
of Maven 3 is simply unacceptable.
* OSGi bundle export data management is tedious
Editing each of our pom.xml files to mark a new release is
difficult enough as it is. Editing every MANIFEST.MF file to
list our exported packages and their current version number is
something a machine should do, not a human. Yet the Tycho OSGi
way unfortunately demands that a human do this work.
* OSGi bundle import data management is tedious
There isn't a way in the MANIFEST.MF file format to reuse the
same version tags across all of our imports, but we want to have
a consistent view of our dependencies when we compile JGit.
After wasting more than 2 full days trying to get Tycho to work,
I've decided its a lost cause right now. We need to be chasing down
bugs and critical features, not trying to bridge the gap between
the stable Maven repository format and the undocumented P2 format
used only by Eclipse.
So, switch the build to use Apache Felix's maven-bundle-plugin.
This is the same plugin Jetty uses to produce their OSGi bundle
manifests, and is the same plugin used by the Apache Felix project,
which is an open-source OSGi runtime. It has a reasonable number
of folks using it for production builds, and is running on top of
the stable Maven 2.x code base.
With this switch we get automatically generated MANIFEST.MF files
based on reasonably sane default rules, which reduces the amount
of things we have to maintain by hand. When necessary, we can add
a few lines of XML to our POMs to tweak the output.
Our build artifacts are still fully compatible with Maven 2.x, so
any downstream consumers are still able to use our build products,
without stepping up to Maven 3.x. Our artifacts are also valid as
OSGi bundles, provided they are organized on disk into a repository
that the runtime can read.
With maven-bundle-plugin the build runs offline, as much as Maven
2.x is able to run offline anyway, so we're able to return to a
distributed development environment again.
By generating MANIFEST.MF at the top level of each project (and
therefore outside of the target directory), we're still compatible
with Eclipse's PDE tooling. Our projects can be imported as standard
Maven projects using the m2eclipse plugin, but the PDE will think
they are vaild plugins and make them available for plugin builds,
or while debugging another workbench.
This change also completely removes Tycho from the build.
Unfortunately, Tycho 0.6.0's pom-first dependency resolver is broken
when resolving a pom-first plugin bundle through a manifest-first
feature package, so bundle org.eclipse.jgit can't be resolved,
even though it might actually exist in the local Maven repository.
Rather than fight with Tycho any further, I'm just declaring it
plugina-non-grata and ripping it out of the build.
Since there are very few tools to build a P2 format repository, and
no documentation on how to create one without running the Eclipse
UI manually by poking buttons, I'm declaring that we are not going
to produce a P2 update site from our automated builds.
Change-Id: If7938a86fb0cc8e25099028d832dbd38110b9124
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This makes the jgit command line behave like the C Git implementation
in the respect.
These variables are not recognized in the core, though we add support
to do the overrides there. Hence other users of the JGit library, like
the Eclipse plugin and others, will not be affected.
GIT_DIR
The location of the ".git" directory.
GIT_WORK_TREE
The location of the work tree.
GIT_INDEX_FILE
The location of the index file.
GIT_CEILING_DIRECTORIES
A colon (semicolon on Windows) separated list of paths that
which JGit will not cross when looking for the .git directory.
GIT_OBJECT_DIRECTORY
The location of the objects directory under which objects are
stored.
GIT_ALTERNATE_OBJECT_DIRECTORIES
A colon (semicolon on Windows) separated list of object directories
to search for objects.
In addition to these we support the core.worktree config setting when
the git directory is set deliberately instead of being found.
Change-Id: I2b9bceb13c0f66b25e9e3cefd2e01534a286e04c
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
An extra flag when creating a RefUpdate object allows the
caller to destroy the symref and replace it with an object
ref, a.k.a. detached HEAD.
Change-Id: Ia88d48eab1eb4861ebfa39e3be9258c3824a19db
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The test data is expected to have unix new lines by tests, but it
is converted to crlf on Windows platform (with msys git). As result
DiffFormatterReflowTest tests fail. To prevent this problem,
crlf conversion is disbled for test data related to that test.
Bug: 295077
Change-Id: I67d3ed543fcc38647041896146de12b1781ec6be
Signed-off-by: Constantine Plotnikov <constantine.plotnikov@gmail.com>
Add some tests which make sure that the diff algorithm really behaves in the
promised O(N*D) manner. This tests compute diffs between multiple big chunks
of data, measure time for computing the diffs and fail if the measured times
are off O(N*D) by more than a factor 10
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Change-Id: I8e1e0be60299472828718371b231f1d8a9dc21a7
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Not all of our test cases really require the sample data packs,
and we are better off not using them because its hard to see exactly
what condition a test is testing when looking only at the Java code.
Clarify the dependency by only making the packs available when
there is a real need for it.
Change-Id: Id8a76ee7ee1f7efba585be4bed19a8fb5b3b3585
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This test depends upon the external git binary, and this isn't
really a pure Java test like our module tries to claim itself is.
So we move it out to exttst with other tests that require additional
external resources and/or executable code.
Change-Id: Ic9be0280c8bb50a5768336c64de794eb0a492b3d
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Drop our simple and stupid jgit.sh and instead rely upon Maven
for the command line based build. Maven is relatively simple to
download and install, and doesn't require the entire Eclipse IDE.
To avoid too much refactoring of the current code we reuse the
existing src/ directory within each plugin, and treat each of
the existing OSGI bundles as one Maven artifact.
The command line wrapper jgit.sh no longer works in the uncompiled
state, as we don't know where to obtain our JSch or args4j from.
Developers will now need to compile it with `mvn package`, or run
our Main class from within an IDE which has the proper classpath.
Bug: 291265
Change-Id: I355e95fa92fa7502651091d2b651be6917a26805
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This way we depend upon the MANIFEST.MF to define our classpath
and our build will act more like any other OSGI bundle build.
Change-Id: I9e1f1f5a0bccb0ab0e39e49b75fb400fea446619
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
According the javadoc, and implied by the name of the class, NB
is about network byte order. The purpose of moving the IO only,
and non-byte order related functions to another class is to
make it easier for new contributors to understand that they
can use these functions in general and it's also makes it easier
to understand where to put new IO related utility functions
Change-Id: I4a9f6b39d5564bc8a694b366e7ff3cc758c5181b
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Our project coding conventions do not want trailing whitespace at
the end of a source code line. Configure Eclipse to automatically
remove them when saving any Java source file.
Change-Id: I9701366b3b1240879761b30556e6ff416e969e1d
Reviewed-by: Mykola Nikishov <mn@mn.com.ua>
Reviewed-by: Alex Blewitt <alex.blewitt@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
As discussed on the egit-dev mailing list, we prefer not to have
trailing whitespace in our source code. Correct all currently
offending lines by trimming them.
Change-Id: I002b1d1980071084c0bc53242c8f5900970e6845
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This way dependencies are described by the MANIFEST.MF,
and the same build tools can be used to compile the tests.
Change-Id: I4dc926148410ecbadcf71b9474aeeb509691aa32
All 3rd party dependencies must come from orbit to comply
with Eclipse development process.
Change-Id: Ia43892ab6d0169f8335c1a41b37e8c12e94cafe2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
When reading commits the system default encoding was used if no
encoding was specified in the commit. The patch modifies the test
to add a check that commit message was encoded correctly (the
test fails on old implementation if system encoding is not UTF-8)
and fixes Commit.decode() method to use UTF-8 if encoding is not
specified in the commit object.
Change-Id: I27101da3c2eb6edd0c4a9e4c0523e48b286e3cd5
Signed-off-by: Constantine Plotnikov <constantine.plotnikov@gmail.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In the pre-historic commit 6d87484b4dee5671a38e64a8e4990dff40a4874f
two tests became identical. Remove one of them.
Change-Id: I6182ecd4db0162d87a5f4577005b2bf4d5e8c89f
Signed-off-by: Robin Rosenberg <robin.rosenberg@dewire.com>
Some applications may wish to modify an int list.
Bug: 291083
Eclipse-CQ: 3559
Change-Id: Iea871443ec661230aec92397229f1eda6c74216f
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Only one test class actually needs this function, so instead of
us inheriting it down into every test, move it to that one class.
Change-Id: I5700ca48df4177153f2b3861dec7c538c621e775
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This test doesn't work because it requires a pack file which we have
lost to the ages. We couldn't include it because the pack was actually
a copy of the GPL'd C git.git project, and was there to test some sort
of corner case that the test never documented properly.
Change-Id: I282ee1c6a637a8654df93a3847507a6c60e4cfab
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Honor the configuration parameter core.logAllRefUpdates when writing
reflogs. Instead of writing reflog entries always only write
reflogs if this parameter is set to true or if the corresponding
file in the <git-dir>/logs directory already exists. In other words:
if you are updating a ref and this parameter is set to false and
there is no file corresponding to your ref in the <git-dir>/logs
folder then no reflog will be written.
This is a fix for the issue http://code.google.com/p/egit/issues/detail?id=4
Change-Id: I908e4c77e3630dc3223b2d2a47cb4534dbe4ed42
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Christian Halstrick <christian.halstrick@sap.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Per CQ 3448 this is the initial contribution of the JGit project
to eclipse.org. It is derived from the historical JGit repository
at commit 3a2dd9921c8a08740a9e02c421469e5b1a9e47cb.
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Shawn O. Pearce
Robin Rosenberg
Shawn O. Pearce
Christian Halstrick
Constantine Plotnikov
Mykola Nikishov
Alex Blewitt
Chris Aniszczyk
Matthias Sohn
Johannes Schindelin
Git Development Community