Rather than relying on the repository's ability to give us the
local file path for a loose object, just pass its inflated form to
the ObjectInserter for the repository. We have to recompress it,
which may slow down fetches, but this is the slow dumb protocol.
The extra cost to do the compression locally isn't going to be a
major bottleneck.
This nicely removes the nasty part about computing the object
identity by hand, allowing us to instead rely upon the inserter's
internal computation. Unfortunately it means we might store a loose
object whose SHA-1 doesn't match the expected SHA-1, such as if the
remote repository was corrupted. This is fairly harmless, as the
incorrectly named object will now be stored under its proper name,
and will eventually be garbage collected, as its not referenced by
the local repository.
We have to flush the inserter after the object is stored because
we aren't sure if we need to read the object later, or not.
Change-Id: Idb1e2b1af1433a23f8c3fd55aeb20575e6047ef0
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The WindowCache is an implementation detail of PackFile and how its
used by ObjectDirectory. Lets start to hide it and replace the public
API with a more generic concept, ObjectReader.
Because PackedObjectLoader is also considered a private detail of
PackFile, we have to make PackWriter temporarily dependent upon the
WindowCursor and thus FileRepository and ObjectDirectory in order to
just start the refactoring. In later changes we will clean up the
APIs more, exposing sufficient support to PackWriter without needing
the file specific implementation details.
Change-Id: I676be12b57f3534f1285854ee5de1aa483895398
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Not every object storage system will have the concept of alternate
object databases to search, and even if they do, they may not have
the notion of fast-access / slow-access split like we do within
the ObjectDirectory code for pack files and loose objects.
Push all of that down below the generic API so that it is a hidden
detail of the ObjectDirectory and its related supporting classes.
Change-Id: I54bc1ca5ff2ac94dfffad1f9a9dad7af202b9523
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some newer style APIs are updated to use the newer ObjectInserter
interface instead of the now deprecated ObjectWriter. In many of
the unit tests we don't bother to release the inserter, these are
typically using the file backend which doesn't need a release,
but in the future should use an in-memory HashMap based store,
which really wouldn't need it either.
Change-Id: I91a15e1dc42da68e6715397814e30fbd87fa2e73
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The ObjectInserter API permits ObjectDatabase implementations to
control their own object insertion behavior, rather than forcing
it to always be a new loose file created in the local filesystem.
Inserted objects can also be queued and written asynchronously to
the main application, such as by appending into a pack file that
is later closed and added to the repository.
This change also starts to open the door to non-file based object
storage, such as an in-memory HashMap for unit testing, or a more
complex system built on top of a distributed hash table.
To help existing application code port to the newer interface we
are keeping ObjectWriter as a delegation wrapper to the new API.
Each ObjectWriter instances holds a reference to an ObjectInserter
for the Repository's top-level ObjectDatabase, and it flushes and
releases that instance on each object processed.
Change-Id: I413224fb95563e7330c82748deb0aada4e0d6ace
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A repository implementation might support storing configurations
on a non-file storage system, so widen the return value to be any
type of configuration.
Change-Id: If9a0928f4b3ef29a24d270b0ce585a6e77f6fac6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When the surrounding code is already heavily based upon the
assumption that we have a FileRepository (e.g. because it
created that type of repository) keep the type around and
use it directly. This permits us to continue to do things
like save the configuration file.
Change-Id: Ib783f0f6a11acd6aa305c16d61ccc368b46beecc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
To support other storage models other than just the local filesystem,
we split the Repository class into a nearly abstract interface and
then create a concrete subclass called FileRepository with the file
based IO implementation.
We are using an abstract class for Repository rather than the much
more generic interface, as implementers will want to inherit a large
array of utility functions, such as resolve(String). Having these in
a base class makes it easy to inherit them.
This isn't the final home for lib.FileRepository. Future changes
will rename it into storage.file.FileRepository, but to do that we
need to also move a number of other related class, which we aren't
quite ready to do.
Change-Id: I1bd54ea0500337799a8e792874c272eb14d555f7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the working tree isn't available, it doesn't make any sense to
obtain the merge heads, or the buffered commit message. The
repository shouldn't have a partial merge state to read. Throw back
the same exception we do when invoking getWorkDir() on a bare
repository instance.
Change-Id: I762c55890b7fe272a183da583f910671d1cadf71
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This permits us to leave the implementation of these methods here in
the Repository class, but later refactor how the directory is accessed
into a subclass.
Change-Id: I5785b2009c5b7cca0fb070a968e50814ce847076
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A bare repository cannot be checked out, committed to, etc. as it
doesn't have a working directory. Define this as a state since the
state enumeration exists only to describe how a working directory
can be modified.
Change-Id: I0a299013c6e42fef6cae3f6a9446f8f6c8e0514a
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This better matches with the other configuration variable,
'userConfig', and helps to make it clear what config object
we are dealing with.
Change-Id: I2c585649aecc805e8e66db2f094828cd2649e549
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Change the Repository API to use straight-up FileBasedConfig.
This lets us remove the subclass RepositoryConfig and stop having
a specialized configuration type for repository, letting us instead
focus the config type heirarchy on type-of-storage rather than use.
Change-Id: I7236800e8090624453a89cb0c7a9a632702691c6
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of using the internal field directly to access references
or objects, use the getter method to obtain the proper type of
database, and follow down from there. This permits us to later
do a refactoring that makes those methods abstract and strips the
field out of the Repository class, moving it into a concrete base
class that is more storage implementation specific.
Change-Id: Ic21dd48800e68a04ce372965ad233485b2a84bef
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This method doesn't need to be synchronized, as its only a proxy to
create(boolean), which is the real worker. While we are touching
it try to improve the Javadoc and whitespace nearby.
Change-Id: Ibdddec6e518ca6d7439cfad90fedfcdc2d6b7a2e
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This permits the repository implementation to know what its
alternates concept means, and avoids needing to expose finer details
about the ObjectDatabase to network code like the RefAdvertiser.
Change-Id: Ic6d173f300cb72de34519c7607cf7b0ff3ea6882
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
In the future getDirectory() can return null. Avoid an NPE here by
refactoring the code to support conditionally skipping a check for
the properties file in the repository directory, falling to only
the user's ~/ file location.
Change-Id: I76f5503d4063fdd9d24b7c1b58e1b09ddf1a5670
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If the destination repository doesn't use an ObjectDirectory to
store its objects, we can't download to the object directory.
Instead pull the pack-*.idx files down to temporary files in the
JVM's default temporary directory.
Change-Id: Ied16bc89be624d87110ba42ba52d698a6ea7d982
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
upload-pack and receive-pack take a git repository as an argument,
but its a lenient path format. Locate the repository and open it.
Change-Id: I4b377e57b28ba3b1717c13d9ab51a602de1ad257
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When reading or locking the index of a repository, we need to use
the index file specified by the repository, to ensure we correctly
honor what the repository was configured with.
Change-Id: I5be366ce32d7923b888dc01d19335912b01b7c4c
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Its not strictly required that we sort topologically in order to
produce a valid pack file. This was just something that Linus
thought would be a good idea to do. In practice its not that
important for most repositories. Local file IO quickly falls
out of the pattern that topological sorting provides any sort
of benefit for, so expending extra resources to enforce it when
we make a pack isn't really worth it.
I'm removing this sort in the pipeline because later changes
would support really efficient COMMIT_TIME_DESC sorting on a
non-file storage system, but TOPO sorting would be a bit more
ugly to run, due to the in-memory delays it imposes.
Change-Id: I0121453461c2140c6917cb10c6df584eb47e5795
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
If UploadPack invokes flush() on the output stream we pass it, its
most likely the progress messages coming down the side band stream.
As pack generation can take a while, we want to push that down
at the client as early as we can, to keep the connection alive,
and to let the user know we are still working on their behalf.
Ensure we dump the temporary buffer whenever flush() is invoked,
otherwise the messages don't get sent in a timely fashion to the
user agent (in this case, git fetch).
We specifically don't implement flush() for ReceivePack right now,
as that protocol currently does not provide progress messages to
the user, but it does invoke flush several times, as the different
streams include '0000' type flush-pkts to denote various end points.
Change-Id: I797c90a2c562a416223dc0704785f61ac64e0220
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We want to eventually get rid of the mapCommit, mapTree APIs on
Repository and force everyone into the faster parsers that exist
in RevWalk. Rewriting resolve in terms of the faster parsers is
a good first step.
It actually simplifies the code a bit, as we no longer need to
keep track of an ObjectId and an Object (the parsed form), since
all RevObjects implicitly have their ObjectId readily available.
Change-Id: I4d234630195616e2c263e7e70038b55a1be4e7a3
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Instead of peeling things by hand in application level code, defer
the peeling logic into RevWalk's new peel utility method.
Change-Id: Idabd10dc41502e782f6a2eeb56f09566b97775a8
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We already have these objects parsed and cached in our object pool.
We shouldn't be looking them up via the legacy mapObject API, but
instead can use the pool and the faster parsing routines available
through the RevWalk that we extend.
While we are here fixing the code, lets also correct the tag date
sorting to accept tags that have no tagger identity, because they
were created before Git knew how to store that field.
Change-Id: Id49a11f6d9c050c82b876e5e11058840c894b2d7
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Rather than relying on the helpers in RepositoryConfig to get
these objects, obtain them directly through the Config API.
Its only slightly more verbose, but permits us to work with the
base Config class, which is more flexible than the highly file
specific RepositoryConfig.
This is what I really meant to do when I added the section parser
and caching support to Config, we just failed to finish updating
all of the call sites.
Change-Id: I481cb365aa00bfa8c21e5ad0cd367ddd9c6c0edd
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
We don't have to assume/depend on RepositoryConfig here, these
two tests can use higher level versions of the class and still
come up with the same test. That frees us up to do some changes
to the RepositoryConfig API.
Change-Id: Ia7b263c8c5efa3fae1054416d39c546867288132
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Long ago we stopped supporting the core.legacyHeaders variable,
as JGit (like C Git) stopped creating the new pack-style loose
objects, rendering this variable pointless. The test is still
valid, it proves we write the standard loose object format for
a commit, but the variable assignment has no impact on the test
so drop it from the code.
Change-Id: I051336ada23033c05e86bbff73ae5d78a37b1640
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Under smart HTTP the biDirectionalPipe flag is false, and we return
back immediately at this point in the negotiation process. There is
no need to flush the stream to the client, the request is over and
it will be automatically flushed out by the higher level servlet
that invoked us. Avoiding flush here allows us to only use flush
after a progress message is sent during pack generation.
Change-Id: Id0c8b7e95e3be6ca4c1b479e096bed6b0283b828
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This simplifies the PackIndex code, which is trying to quickly copy
an existing ObjectId into a MutableObjectId. Rather than having
the PackIndex violate the ObjectId's internals, expose a copy from
function similar to the other ones for copying from raw byte arrays
or hex formatted strings.
Change-Id: I142635cbece54af2ab83c58477961ce925dc8255
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Storage systems can use these implementations to compare a passed
AnyObjectId with a stored representation of an ObjectId in the
canonical network byte order format. This can be useful to do a
binary search, or just linear scan, over an encoded storage file.
Change-Id: I8c72993c4f4c6e98d599ac2c9867453752f25fd2
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
An implementation might prefer to use the RefList type here, and
RefList is part of our public API. Expose the constructor so callers
who have a RefList can take advantage of the existing sorting.
Change-Id: I545867f85aa2c479d2d610024ebbe318144709c8
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
When we finally move RefDirectory to the new storage.file package,
its associated RefDirectoryUpdate will need visiblity to this
constructor in order to initialize itself. This is true of any
other repository implementation, so make it protected rather than
package level visible.
Change-Id: If838aec9baeb80ee2f12dcbca717657c725a9242
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Repository implementations outside of .lib need to be able to
create these events and deliver them to listening application code.
Expose and document the constructors so that they are visible when
we move FileRepository into storage.file.FileRepository.
Change-Id: I7fb6e8f4f5fdab683c5ebb5267673aa6d5b560bb
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
A Git reference name must never end with ".lock", as it would
confuse any existing C client that tries to obtain a clone of the
repository over the network. Even if the repository isn't on a
local filesystem, it still should ban that suffix.
Because I plan to move LockFile to storage.file and make it a private
implementation detail of the local file system storage model,
we can't rely on its package level SUFFIX field here. Making it
public probably won't work long-term either, as I also plan to
pull storage.file into its own separate project that depends on
the core library.
So, just inline the constant here. Its as foribidden as ":" is.
Change-Id: If85076861baeacc183b82696375a13e935ba8836
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
This stream was used only to determine how many bytes had been
written thus far. Except we're always dumping it into a simple
ByteArrayOutputStream, which also knows that. Drop the dependency
on the pack stream and use ByteArrayOutputStream directly.
This lets us later move this test into the new storage.file
package without dragging along the pack stream that is an internal
implementation detail of PackWriter, which is more general than
just the file storage layer.
Change-Id: I291689c0b1ed799270c213ee73b710b2637fb238
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Setting this value is pointless, because its automatically set
by the refs.newUpdate call that created the update operation.
The API is protected by default, because application level code,
including this test, should not be calling it.
Change-Id: I8867a4e8007892e2bd44a05d7dec619081081943
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
The mapCommit API is being deprecated because it doesn't run very
fast. Leaving tests around to test how fast it is relative to C Git
isn't instructive. Remove them, which should help aid the transition
away from the mapCommit API.
Change-Id: I27e1c844610d7da5b2c44b33a00602706973c9cc
Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
Some sources had dos line endings. Also configure all projects to use
unix line endings and UTF-8 text encoding.
Change-Id: I8fc9a1dbb219ffa91d1b3011b3b11b7e48e74ca7
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Modifications to various classes in order to allow serialization
for use of JGit in Hudson's git plugin.
Change-Id: If088717d3da7483538c00a927e433a74085ae9e6
If a repository is "bare", it currently still returns a working directory.
This conflicts with the specification of "bare"-ness.
Bug: 311902
Change-Id: Ib54b31ddc80b9032e6e7bf013948bb83e12cfd88
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
Currently, there is no way to read the content
of the Git Configuration in a way that would
allow to list all configured values generically.
This change extends the Config class in such a
way as to being able to get a list of sections and
to get a list of names for any given section or
subsection.
This is required in able to implement proper
configuration handling in EGit (show all the
content of a given configuration similar to
"git config -l").
Change-Id: Idd4bc47be18ed0e36b11be8c23c9c707159dc830
Signed-off-by: Mathias Kinzler <mathias.kinzler@sap.com>
* changes:
git-servlet: Fix comparing uploadFactory with the wrong DISABLED instance
Prefer static inner classes
Override equals for SwingLane since super class PlotLane defines it
Make sure a Stream is closed upon errors in IpLogGenerator
Make constant static in RebuildCommitGraph
Make inner classes static in http code
Cache filemode in GitIndex
Remove unused parent field in PlotLane
Removed unused repo field in WorkDirCheckout
Extend DiffFormatter API to simplify styling
Shawn O. Pearce
Matthias Sohn
Andrew Bayer
Mathias Kinzler
Chris Aniszczyk
Robin Rosenberg