Add a storage implementation storing large objects in Amazon S3.
The AmazonS3Repository pre-signs download and upload requests.
AWS access and secret key are expected to be in the
$HOME/.aws/credentials file in the following format:
[default]
accessKey = ...
secretKey = ...
Use AWS version 4 request signing [1] because it is more secure and
supported by all regions. The version 3 signing is not supported in
newer regions.
In follow up changes we should:
- implement getVerifyAction() and do actual verification. Subclasses of
S3Repository can implement caching for object meta data (size) in order
to avoid extra roundtrips to S3. Verification should ensure that meta
data store and content of S3 storage are in sync
- HEAD request used in S3Repository.getSize() seems to always return
Content-length 0 in contrast to the documentation [2]. So getSize() does
detect if the object exists in S3 or not but in case the object exists
it always returns size 0
[1] http://docs.aws.amazon.com/general/latest/gr/signature-version-4.html
[2] https://forums.aws.amazon.com/thread.jspa?threadID=223616
Change-Id: Ic47f094928a259e5264c92b3aacf6d90210907a8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Signed-off-by: Sasa Zivkov <sasa.zivkov@sap.com>
Capture the internal "want X not valid" state as a specific subclass
of PackProtocolException, allowing this to be more easily identified
in server stack traces and wrapper application code.
Change-Id: I4b1adb7497f396432da420b0f600ad25a261f912
If the client sends a SHA-1 that the server does not recognize echo
this back to the client with an explicit error message instead of
the generic "internal server error".
This was always the intent of the implementation but it was being
dropped on smart HTTP due to the UploadPackServlet catching the
PackProtocolException, discarding the buffered message UploadPack
meant to send, and sending along a generic message instead.
Change-Id: I8d96b064ec655aef64ac2ef3e01853625af32cd1
* stable-4.2:
Don't use deprecated LockFile constructor
Fix warnings about unchecked conversion of MergeResult
MockServletConfig: Fix warning about unchecked conversion of Enumeration
HugeFileTest: Make Git a class member and open in try-with-resource
Suppress "unchecked cast" warnings related to UploadPackFactory.DISABLED
DiffAlgorithms: Fix warnings about variable hiding
DirCacheBasicTest: Open ObjectInserter.Formatter in try-with-resource
DirCacheBuilderIteratorTest: Open TreeWalk in try-with-resource
DirCacheCGitCompatabilityTest: Open TreeWalk in try-with-resource
DirCacheCheckoutMaliciousPathTest: Open Git and RevWalk in t-w-r
DirCacheIteratorTest: Open TreeWalk instances in try-with-resource
ForPathTest: Open TreeWalk in try-with-resource
GitConstructionTest: Open Git instance in try-with-resource
IndexDiffTest: Open Git instances in try-with-resources
ManifestParserTest: Don't use deprecated StringBufferInputStream
InMemoryRepository: Remove unused RevWalk from batch method signature
IndexModificationTimesTest: Open Git instances in try-with-resource
InterIndexDiffFilterTest: Open TreeWalk in try-with-resource
LockFileTest: Open Git instance in try-with-resource
JGit v4.1.2.201602141800-r
MergeCommandTest: Use JUnit's assume to check preconditions
MergeCommandTest: Open Git instances in try-with-resource
Change-Id: Ie5dba6b9132a29e86958a04fa2b76465bcd2c6b5
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
There's only one test method in this module and it's quite long, so
rather than using a try-with-resource and having to indent a huge
block of existing code, make the Git a member variable that gets
initialised and closed in @Before and @After annotated methods.
The methods are named 'before' and 'after' rather than the conventional
'setUp' and 'tearDown' so as not to conflict with the names of the
existing methods in LocalDiskRepositoryTestCase.
Change-Id: I5a4a9b59f244c450dbcae9fdde7d9e0f0cd24e6f
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Local variables/parameters named 'db' and 'cmp' were hiding class
member variables of the same name.
Change-Id: I98b770587aaf73744a93e6a3ee33d131a9fa91e9
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Replace it with ByteArrayInputStream as suggested in [1].
[1] http://stackoverflow.com/a/2219543/381622
Change-Id: I5ca8d721a756a82ea5f5687a20555303eb1dfc18
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The RevWalk given in the arguments is not used. According to the
comment at the top of the method, a new RevWalk is intentionally
used in the implementation.
Remove the unused argument.
Change-Id: Iec81a1341d5bf377801475845b96a465753096ef
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Remove all proxy rules, that were introduced to allow to build Gerrit
with hijacked JGit cell. New approach suggested in: [1], that emulates
real JGit project structure in its own cell, makes them unnecessary.
Add :all rule, that build all artifacts and packages them in zip file.
Add shell binary :jgit_bin rule, that allows to execute JGit binary
from with buck run command, e.g.:
$ buck run jgit_bin status
$ buck run jgit_bin -- --version
* [1] https://gerrit-review.googlesource.com/74859
Change-Id: Idf9ecb783cbd2b9984d4118047968f1f5204d642
Signed-off-by: David Ostrovsky <david@ostrovsky.org>
* stable-4.2:
NoteMapTest: Open TreeWalk instances in try-with-resource
ObjectDirectoryTest: Fix warnings about variable hiding
PackWriterTest: Open RevWalk in try-with-resource
PatchIdDiffFormatterTest: Open Git and PatchIdDiffFormatter in try-with-resource
PathSuffixFilterTest: Open TreeWalk in try-with-resource
PostOrderTreeWalkTest: Open TreeWalk in try-with-resource
PullCommandWithRebaseTest: Open RevWalk in try-with-resource
PushCommandTest: Open Git instances in try-with-resource
RacyGitTests: Open NameConflictTreeWalk in try-with-resource
RecursiveMergerTest: Open TreeWalk and BufferedReader in try-with-resource
ReflogConfigTest: refactor commit method to avoid variable hiding
Update .mailmap
RefDirectoryTest: Fix warning about member variable hiding
ReflogResolveTest: Open Git instances in try-with-resource
ReflogTest: Open Git instances in try-with-resource
RepoCommandTest: Open Git instances in try-with-resource
Change-Id: I7964b699396629e31a9cc5600aedcf4be4e659a8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Using the assume method, instead of just returning, will cause the
test to be marked as skipped rather than passed on systems where
the precondition is not satisfied.
Change-Id: I13672371f6cd3c481a0a6247e0eaed3aac6d766e
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The variable and parameter named 'db' were hiding class members
with the same name.
Change-Id: I27017afdc5f49c38c6f5be494e7a21239ea601a7
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Also remove unnecessary nesting in test methods.
Change-Id: Id59f8403c0a7b38ebb6b3a24814257cd59ea575d
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
The author and committer parameters were hiding class members of
the same name.
Since the passed in PersonIdent instances were being created from
the class members anyway, remove the parameters and instead create
the PersonIdent instances inline in the method.
Change-Id: I66b057df388835d57f332fdcbadb8a9f4e1094a4
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
- List names alphabetically by Firstname Surname
- Replace tabs with spaces
- Add missing name
Change-Id: Ib320dad316e0f6e3e95363c4d6a64744032db857
Signed-off-by: David Pursehouse <david.pursehouse@sonymobile.com>
Sasa Zivkov
Matthias Sohn
Shawn Pearce
David Pursehouse
David Ostrovsky
Saša Živkov