* stable-5.3:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Change-Id: I12002dbfed9dff14fc6d2df9787d92eab5b1fa78
* stable-5.2:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Change-Id: I790098be00ff3f9b8278d54ae4fb7c11311816e9
* stable-5.1:
Fix string format parameter for invalidRefAdvertisementLine
WindowCache: add metric for cached bytes per repository
pgm daemon: fallback to user and system config if no config specified
WindowCache: add option to use strong refs to reference ByteWindows
Change-Id: I741059a1d0d5950ab5bc16ec70352655ee926a24
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The externalized error message added in f4fc640 ("BasePackConnection:
Check for expected length of ref advertisement", Dec 18, 2019) uses a
malformed string format. Since there is only one formatting argument,
it should be referenced with '{0}' rather than '{1}'.
Change-Id: Ibda864dfb0bb902fe07ae4bba73117b212046e8a
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Since ObjectDatabase and PackFile don't know their repository use the
packfile's grand-grand-parent directory as an identifier for the
repository the packfile resides in.
Remove metric for a repository if the number of cached bytes for the
repository drops to 0 in order to ensure the map of cached bytes per
repository doesn't contain repositories which have no data cached in the
WindowCache.
Change-Id: I969ab8029db0a292e7585cbb36ca0baa797da20b
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
If a config file is passed via option --config-file then use only the
options defined in that file. This helps to concisely configure the
daemon without side effects from global and system level git configs.
Otherwise fallback to user and system level configs.
Change-Id: I242de248f257579874ad0bfe4882a22502353b1f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Java GC evicts all SoftReferences when the used heap size comes close to
the maximum heap size. This means peaks in heap memory consumption can
flush the complete WindowCache which was observed to have negative
impact on performance of upload-pack in Gerrit.
Hence add a boolean option core.packedGitUseStrongRefs to allow using
strong references to reference packfile pages cached in the WindowCache.
If this option is set to true Java gc can no longer flush the
WindowCache to free memory if the used heap comes close to the maximum
heap size. On the other hand this provides more predictable performance.
Bug: 553573
Change-Id: I9de406293087ab0fa61130c8e0829775762ece8d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Replace usage of ArrayIndexOutOfBoundsException in treewalk
Add config constants for WindowCache configuration options
Change-Id: I17da3c5183eca536aa2be3972bc5df45c9d75f1b
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Using exceptions during normal operations - for example with the
desire of expanding an array in the failure case - can have a
severe performance impact. When exceptions are instantiated,
a stack trace is collected. Generating stack trace can be expensive.
Compared to that, checking an array for length - even if done many
times - is cheap since this is a check that can run in just a
handful of CPU cycles.
Change-Id: Ifaf10623f6a876c9faecfa44654c9296315adfcb
Signed-off-by: Patrick Hiesel <hiesel@google.com>
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I1b560b36d169cfa02cc5450ad0fa0bd85f9f42d8
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Fix API problem filters
Fix unclosed resource warning in SmartOutputStream
JschConfigSessionFactory: fix boxing warning
SshSupport#runSshCommand: don't throw exception in finally block
Don't override already managed maven-compiler-plugin version
Remove unused import from CreateFileSnapshotBenchmark
Remove duplicate ignore_optional_problems entry in .classpath
Update maven-site-plugin used by benchmark module to 3.8.2
Add dependency to enable site generation for benchmark module
Ignore warnings for generated source code in
org.eclipse.jgit.benchmark
Fix MBean registration
Enhance WindowCache statistics
Change-Id: I67a07d92718188bdf7f8a13b83e9f538ecf4b22f
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The benchmark module currently has no parent, adjust the version used
here to the one used by all the other jgit Maven modules.
Change-Id: I8807a694fe23f8f131d1d22a58a3e18874d756cc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The source code in the folder .apt_generated is generated by the JMH
code generator, so there's no point in raising any warnings as this
could only be fixed in the upstream code generator.
Change-Id: I882888e7bf924f9ae74182598fcb91671a5c9818
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Add the following statistics
- cache hit count and hit ratio
- cache miss count and miss ratio
- count of successful and failed loads
- rate of failed loads
- load, eviction and request count
- average and total load time
Use LongAdder instead of AtomicLong to implement counters in order to
improve scalability.
Optionally expose these metrics via JMX, they are registered with the
platform MBean server if the config option jmx.WindowCacheStats = true
in the user or system level git config.
Bug: 553573
Change-Id: Ia2d5246ef69b9c2bd594a23934424bc5800774aa
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.3:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: I21878c42fd9abf7d858b534300df0fffe4bad431
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.2:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: Ie5e66f2e26c057c81101b0d110f91ea479eb362d
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.1:
Bazel: Use java_plugin and java_binary from @rules_java in jmh.bzl
Bazel: Add missing newlines at end of BUILD files
Change-Id: Ic1dc84a6c4a1cf430d329627642583fd6e0d0eaa
Signed-off-by: David Pursehouse <david.pursehouse@gmail.com>
* stable-5.3:
Run JMH benchmarks using bazel
Benchmark for creating files and FileSnapshots
Implement benchmark for looking up FileStore of a given Path
JMH benchmark for SimpleLruCache
Update API problem filters
Remove unused API problem filters
Silence API errors for new API added since 5.1.0
Change-Id: Ib735c4039e24ec8b045ae2cc81df1e9e5c9fa996
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.2:
Run JMH benchmarks using bazel
Benchmark for creating files and FileSnapshots
Implement benchmark for looking up FileStore of a given Path
JMH benchmark for SimpleLruCache
Update API problem filters
Remove unused API problem filters
Silence API errors for new API added since 5.1.0
Change-Id: I00615958ab6fbdff601e87a9792aba5606cda12a
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
* stable-5.1:
Run JMH benchmarks using bazel
Benchmark for creating files and FileSnapshots
Implement benchmark for looking up FileStore of a given Path
JMH benchmark for SimpleLruCache
Remove unused API problem filters
Silence API errors for new API added since 5.1.0
Change-Id: If91c55a192d3b2c441d9c8d414f2e24a7261b1b6
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
The benchmarks can be built and run using bazel by running:
$ bazel run //org.eclipse.jgit.benchmarks:benchmarks
Change-Id: I6679750eaa0f2be30ed9d45036e013b0ea4fcc86
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Results on Mac OS 10.14.6 using APFS:
Result "org.eclipse.jgit.benchmarks.LookupFileStoreBenchmark.testLookupFileStore":
44583.277 ±(99.9%) 2666.096 ns/op [Average]
(min, avg, max) = (43707.073, 44583.277, 45320.685), stdev = 692.377
CI (99.9%): [41917.181, 47249.373] (assumes normal distribution)
Secondary result
"org.eclipse.jgit.benchmarks.LookupFileStoreBenchmark.testLookupFileStore:·stack":
Stack profiler:
....[Thread state
distributions]........................................................
100.0% RUNNABLE
....[Thread state:
RUNNABLE].............................................................
42.5% 42.5% sun.nio.fs.UnixNativeDispatcher.realpath0
37.9% 37.9% sun.nio.fs.UnixNativeDispatcher.stat0
11.0% 11.0% sun.nio.fs.BsdNativeDispatcher.getfsstat
7.6% 7.6% sun.nio.fs.BsdNativeDispatcher.fsstatEntry
0.7% 0.7% sun.nio.fs.BsdNativeDispatcher.endfsstat
0.1% 0.1% sun.nio.fs.UnixPath.initOffsets
0.0% 0.0% sun.nio.fs.UnixFileAttributes.get
0.0% 0.0% java.util.zip.Inflater.inflateBytes
0.0% 0.0% sun.misc.Unsafe.compareAndSwapInt
0.0% 0.0% sun.nio.fs.BsdFileStore.findMountEntry
This shows that FS.attrCacheByPath should be useful to reduce overhead
of looking up the FileStore of a given file.
Change-Id: I7213086b42e0453f0ee149660d507dac2a4644cc
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
See [1] for JMH documentation and [2] how to use JMH in Eclipse.
The benchmarks pom currently cannot use the JGit parent pom due to an
ecj bug [3] regarding annotation processing. Hence for now do not
inherit from the JGit parent pom and copy the compiler plugin
configuration for javac from the parent pom.
After running the Maven build the benchmark can be run using Maven:
$ java -jar org.eclipse.jgit.benchmarks/target/benchmarks.jar
or in Eclipse by running the main method of the SimpleLruCacheBenchmark
class.
[1] https://openjdk.java.net/projects/code-tools/jmh/
[2] http://alblue.bandlem.com/Page/3/index.html
[3] https://eclip.se/532029
CQ: 20517
CQ: 20518
Change-Id: Idca8a9e0980f0b8a9c741c4c9e97d03c62f07c8d
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
- filter errors for new APIs added in service release
- remove unused filters
Change-Id: Ifbf532b8a3c46d4ed78a38f6c75073a072b7f669
Signed-off-by: Matthias Sohn <matthias.sohn@sap.com>
Matthias Sohn
David Pursehouse
Patrick Hiesel