jgit.storage.dht is a storage provider implementation for JGit that
permits storing the Git repository in a distributed hashtable, NoSQL
system, or other database. The actual underlying storage system is
undefined, and can be plugged in by implementing 7 small interfaces:
* Database
* RepositoryIndexTable
* RepositoryTable
* RefTable
* ChunkTable
* ObjectIndexTable
* WriteBuffer
The storage provider interface tries to assume very little about the
underlying storage system, and requires only three key features:
* key -> value lookup (a hashtable is suitable)
* atomic updates on single rows
* asynchronous operations (Java's ExecutorService is easy to use)
Most NoSQL database products offer all 3 of these features in their
clients, and so does any decent network based cache system like the
open source memcache product. Relying only on key equality for data
retrevial makes it simple for the storage engine to distribute across
multiple machines. Traditional SQL systems could also be used with a
JDBC based spi implementation.
Before submitting this change I have implemented six storage systems
for the spi layer:
* Apache HBase[1]
* Apache Cassandra[2]
* Google Bigtable[3]
* an in-memory implementation for unit testing
* a JDBC implementation for SQL
* a generic cache provider that can ride on top of memcache
All six systems came in with an spi layer around 1000 lines of code to
implement the above 7 interfaces. This is a huge reduction in size
compared to prior attempts to implement a new JGit storage layer. As
this package shows, a complete JGit storage implementation is more
than 17,000 lines of fairly complex code.
A simple cache is provided in storage.dht.spi.cache. Implementers can
use CacheDatabase to wrap any other type of Database and perform fast
reads against a network based cache service, such as the open source
memcached[4]. An implementation of CacheService must be provided to
glue this spi onto the network cache.
[1] https://github.com/spearce/jgit_hbase
[2] https://github.com/spearce/jgit_cassandra
[3] http://labs.google.com/papers/bigtable.html
[4] http://memcached.org/
Change-Id: I0aa4072781f5ccc019ca421c036adff2c40c4295
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In order to run the static checks run:
mvn -P static-checks clean install
Change-Id: I14077498a04be986ded123ddbfc97da8f9bc3130
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
It fixes 'Corrupted MAC on input' ssh issue.
CQ: 4728
Change-Id: I7de63cb3482488ac938566706edebee8a1cdad3a
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Set the plugin version in the top-level pom, not the unit test pom.
This ensures the same plugin is used for all JUnit tests within the
overall project.
Drop the include **/*Test.java definition, as its no longer necessary
with the JUnit 4 based test suite. All of the test classes now end
with "Test" and include @Test annotations on the test methods.
Change-Id: Ib2c180bf531e1a97e31979fcc281fa0fc5a1abb3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Jetty 7.1.6 is used because this version is also available in P2.
Change-Id: I410fbca8592cac6e58c651c4d086573820e777a5
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Introduce a http test bundle to make this functionality available for
EGit tests. A simple http server class is provided. The jetty version
was updated to a version that is also available via p2 (needed in EGit
UI tests).
Change-Id: I13bfc4c6c47e27d8f97d3e9752347d6d23e553d4
Signed-off-by: Jens Baumgart <jens.baumgart@sap.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
We need to use findbugs-maven-plugin:2.3.2-SNAPSHOT
since otherwise build fails with maven-3.0 [1], [2].
We should switch to the release version as soon
as this becomes available.
[1] http://www.sonatype.com/people/2010/10/maven-3-0-has-landed/
[2] http://jira.codehaus.org/browse/MFINDBUGS-122
Bug: 327799
Change-Id: I1c57f81cf6f0450e56411881488c4ee754e458e3
Signed-off-by: Chris Aniszczyk <caniszczyk@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The maven 2 build for jgit source bundle didn't create a correct
OSGi version string, instead of
org.eclipse.jgit.source_0.10.0.<timestamp>
the generated OSGi version was
org.eclipse.jgit.source_0.10.0.SNAPSHOT
This caused trouble when trying to install it from p2 repository.
Bug: 327616
Change-Id: Ic27c763ae9a4bcbb5bd6ed9562cd98bb4da3386b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add local changes I missed to push with [1] which broke the JGit
build.
[1] http://egit.eclipse.org/r/#change,1442
Change-Id: I300bfa84c5d8b5128026869b694ef5da7b0d3a4a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Created wrong tags for 0.8.3 hence creating another version.
Change-Id: I4e00bbcffe1cf872e2d7e3f3d88d068701fb5330
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
There is a serious problem with the Maven Javadoc plugin. Please see
http://jira.codehaus.org/browse/MJAVADOC-275
for details. This problem is fixed by using maven-javadoc-plugin V2.7
instead of maven-javadoc-plugin v2.6.1.
This prevents surprises by implicit updates to newer versions.
Change-Id: I06508036d468fa5299ea774e26a73312bb286ec2
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Since the API is changing relative to 0.7.0, we'll call our next
release 0.8.1. But until that gets released, builds from master
will be 0.8.0.qualifier.
Change-Id: I921e984f51ce498610c09e0db21be72a533fee88
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The new plugin contains the bulk of the logic to scan a Git repository,
and query IPZilla, in order to produce an XML formatted IP log for the
requested revision of any Git based project. This plugin is suitable
for embedding into a servlet container, or into the Eclipse workbench.
The command line pgm package knows how to invoke this plugin through
the eclipse-iplog subcommand, permitting storage of the resulting
log as a local XML file.
Change-Id: If01d9d98d07096db6980292bd5f91618c55d00be
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Translate the version qualifier using maven-antrun-plugin since we want
manifest-first and currently cannot rely on Tycho for the JGit build.
Introduce property for Eclipse p2 repository to enable builds against
other Eclipse versions.
Change-Id: I62c4e77ae91fe17f56c5a5338d53828d4e225395
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
No Eclipse support for this project is provided, because the
Jetty project does not publish a complete P2 repository.
Change-Id: Ic5fe2e79bb216e36920fd4a70ec15dd6ccfd1468
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This is a simple HTTP server that provides the minimum server side
support required for dumb (non-git aware) transport clients.
We produce the info/refs and objects/info/packs file on the fly
from the local repository state, but otherwise serve data as raw
files from the on-disk structure.
In the future we could better optimize the FileSender class and the
servlets that use it to take advantage of direct file to network
APIs in more advanced servlet containers like Jetty.
Our glue package borrows the idea of a micro embedded DSL from
Google Guice and uses it to configure a collection of Filters
and HttpServlets, all of which are matched against requests using
regular expressions. If a subgroup exists in the pattern, it is
extracted and used for the path info component of the request.
Change-Id: Ia0f1a425d07d035e344ae54faf8aeb04763e7487
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Since Robin reverted using the maven-bundle-plugin to produce the
OSGi manifest, there is no reason for us to reference it from our
build process anymore.
Also, when Robin reverted the to the Eclipse way of doing things,
we failed to update the ignore files to ignore our generated files
but not ignore our tracked .classpath.
Finally, we cannot delete the MANIFEST.MF file during a Maven build,
as this is once again a source file.
Change-Id: I53f77f2002cb4285f728968829560e835651e188
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Default maven-bundle-plugin behaviour results in use of the same
.SNAPSHOT OSGi bundle version qualifier for all snapshot builds.
This causes problems for eclipse update manager and other consumers
that rely on OSGi bundle metadata to select "newer" or "best
matching" version of jgit bundle.
To solve the problem, maven-bundle-plugin is configured to replace
.SNAPSHOT with build timestamp in format like 20100106-1234.
Change-Id: I0999c7bd68aa2ee74dffaed54a8dc4e1b67cf80d
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Tycho isn't production ready for projects like JGit to be using as
their primary build driver. Some problems we ran into with Tycho
0.6.0 that are preventing us from using it are:
* Tycho can't run offline
The P2 artifact resolver cannot perform its work offline. If the
build system has no network connection, it cannot compile a
project through Tycho. This is insane for a distributed version
control system where developers are used to being offline during
development and local testing.
* Magic state in ~/.m2/repository/.meta/p2-metadata.properties
Earlier iterations of this patch tried to use a hybrid build,
where Tycho was only used for the Eclipse specific feature and P2
update site, and maven-bundle-plugin was used for the other code.
This build seemed to work, but only due to magic Tycho specific
state held in my local home directory. This means builds are not
consistently repeatable across systems, and lead me to believe
I had a valid build, when in fact I did not.
* Manifest-first build produces incomplete POMs
The POM created by the manifest-first build format does not
contain the dependency chain, leading a downstream consumer to
not import the runtime dependencies necessary to execute the
bundle it has imported. In JGit's case, this means JSch isn't
included in our dependency chain.
* Manifest-first build produces POMs unreadable by Maven 2.x
JGit has existing application consumers who are relying on
Maven 2.x builds. Forcing them to step up to an alpha release
of Maven 3 is simply unacceptable.
* OSGi bundle export data management is tedious
Editing each of our pom.xml files to mark a new release is
difficult enough as it is. Editing every MANIFEST.MF file to
list our exported packages and their current version number is
something a machine should do, not a human. Yet the Tycho OSGi
way unfortunately demands that a human do this work.
* OSGi bundle import data management is tedious
There isn't a way in the MANIFEST.MF file format to reuse the
same version tags across all of our imports, but we want to have
a consistent view of our dependencies when we compile JGit.
After wasting more than 2 full days trying to get Tycho to work,
I've decided its a lost cause right now. We need to be chasing down
bugs and critical features, not trying to bridge the gap between
the stable Maven repository format and the undocumented P2 format
used only by Eclipse.
So, switch the build to use Apache Felix's maven-bundle-plugin.
This is the same plugin Jetty uses to produce their OSGi bundle
manifests, and is the same plugin used by the Apache Felix project,
which is an open-source OSGi runtime. It has a reasonable number
of folks using it for production builds, and is running on top of
the stable Maven 2.x code base.
With this switch we get automatically generated MANIFEST.MF files
based on reasonably sane default rules, which reduces the amount
of things we have to maintain by hand. When necessary, we can add
a few lines of XML to our POMs to tweak the output.
Our build artifacts are still fully compatible with Maven 2.x, so
any downstream consumers are still able to use our build products,
without stepping up to Maven 3.x. Our artifacts are also valid as
OSGi bundles, provided they are organized on disk into a repository
that the runtime can read.
With maven-bundle-plugin the build runs offline, as much as Maven
2.x is able to run offline anyway, so we're able to return to a
distributed development environment again.
By generating MANIFEST.MF at the top level of each project (and
therefore outside of the target directory), we're still compatible
with Eclipse's PDE tooling. Our projects can be imported as standard
Maven projects using the m2eclipse plugin, but the PDE will think
they are vaild plugins and make them available for plugin builds,
or while debugging another workbench.
This change also completely removes Tycho from the build.
Unfortunately, Tycho 0.6.0's pom-first dependency resolver is broken
when resolving a pom-first plugin bundle through a manifest-first
feature package, so bundle org.eclipse.jgit can't be resolved,
even though it might actually exist in the local Maven repository.
Rather than fight with Tycho any further, I'm just declaring it
plugina-non-grata and ripping it out of the build.
Since there are very few tools to build a P2 format repository, and
no documentation on how to create one without running the Eclipse
UI manually by poking buttons, I'm declaring that we are not going
to produce a P2 update site from our automated builds.
Change-Id: If7938a86fb0cc8e25099028d832dbd38110b9124
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Changed Tycho version from 0.6.0-SNAPSHOT to 0.6.0 (i.e. release).
SNAPSHOT versions are transient and should only be used for testing
purposes only. Also removed now unnecessary <pluginRepositories/>
element from JGit parent pom.xml file.
Change-Id: Ie386b2dbcba43c1ccec10465978d12d6829c6150
Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Chris Aniszczyk
Shawn O. Pearce
Matthias Sohn
Anatol Pomozov
Robin Rosenberg
Jens Baumgart
Jonathan Gossage
Igor Fedorenko