08335b1032
[Improvement][Task] Improved way to collect yarn job's appIds ( #12197 )
...
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
2 years ago
e6da1ccf81
Add worker-group-refresh-interval in master config ( #12601 )
...
* Add worker-group-refresh-interval in master config
* Set interval cannot smaller than 10s
* Update dolphinscheduler-master/src/main/java/org/apache/dolphinscheduler/server/master/config/MasterConfig.java
Co-authored-by: kezhenxu94 <kezhenxu94@apache.org>
2 years ago
064fec88b0
[Feature-12040][docs] Update Granted Permissions Guide for PR#12048 ( #12587 )
...
* Upload new ui screenshots.
* Update document of Granted Permissions.
* Delete unused images.
2 years ago
207b4e77d3
Fix document styles. ( #12600 )
2 years ago
4a13148bfc
[Bug] [API] Before deleting a worker group, check whether there is environment that reference the worker group. ( #12534 )
...
* [Bug] [API] Before deleting a worker group, check whether there is environment that reference the worker group.
Co-authored-by: houshitao <shitaohou@163.com>
2 years ago
21caec01db
fix doc CI ( #12560 )
2 years ago
63244c8065
Optimized complement documentation ( #12550 )
2 years ago
76a9e6d8a6
Add task executor threads full policy config in worker ( #12510 )
2 years ago
8add9744d4
[doc][feat] New release 2.0.7 ( #12507 )
2 years ago
d46e4659d9
[Improvement][Style] Fix remaining style errors and wildcard imports ( #12412 )
...
* Fix remaining style errors and wildcard imports
* Upgrade spotless to the latest stable version, exclude .github folder from spotless check and fix dead link
* Fix spotless matching scope of markdown files
2 years ago
ba538067f2
[fix] Change the default unix shell executor from sh to bash ( #12180 )
...
Co-authored-by: Kerwin <37063904+zhuangchong@users.noreply.github.com>
2 years ago
1d0d26a416
[fix#12170] If the worker's clock early than master node, will ignore dispatch time. ( #12219 )
...
* add doc to guide user configure synchronized clocks.
2 years ago
7aa8a77fb2
[Migrate][Test] Migrate other UT cases from jUnit 4 to 5, change @Ignore to @Disable
2 years ago
a725614429
[docs] change optional parameters --jar to --jars for Spark ( #12385 ) ( #12386 )
2 years ago
b953cdc559
update english oracle.md ( #12332 )
2 years ago
c50cc997a8
[Doc][Bug] Fix dead link caused by markdown cross-files anchor ( #12357 )
2 years ago
501119bbb6
[doc] Change release process base on new tool ( #12324 )
...
ref: #11478 we already have a new tool for releasing
2 years ago
a973054950
[ci] Milestone and Label are required before the PR merged ( #12222 )
...
To make our release process easier, I plan to mark milestone
and label is required before the PR merged, they use for
* label: In this path we required label with at least on with
'feature|bug|improvement|document|chore' to make our changelog
more convenient
* milestone: For easier to cherry-pick PR, currently some of
our milestone only exists in issue and not exists in related
PR, it may
2 years ago
ef328d9cea
Update ZooKeeper minimum version requirements ( #12284 ) ( #12288 )
2 years ago
c8bd106604
create oracle help.md ( #12218 )
...
* create oracle help.md
* Add files via upload
allow Oracle to choose schema
* move pic under img and add title for oracle.md
2 years ago
9b1ee1516f
[doc] Correct descriptions in glossary.md ( #12282 )
2 years ago
c7435a51ac
[Feature][task-plugins] aws datasync task plugins ( #11865 )
...
* submit datasync task
* add aws software package license
2 years ago
61457d1c2f
[Hotfix][Docs] Fix 404 dead links ( #12261 )
2 years ago
7bdbd62985
[doc] Update slack invitation link ( #12258 )
2 years ago
7beff83e9f
[Doc] Add chmod command after unzip DS tar.gz ( #12210 )
2 years ago
6fc209ab9d
[fix][doc] Correct release export key step ( #12228 )
...
We should only add KEY to release/KEYS instead of
dev/KEYS, according to
https://lists.apache.org/thread/2gppxc04n0r87vq4qy4j3n52zwhqd442
2 years ago
40090d459e
[Improvement] remove log-server and server module ( #12206 )
...
* remove log-server module
* remove dolphinscheduler-server module
2 years ago
aacabc6ec7
Remove workerGroup in registry ( #12217 )
2 years ago
a055cf5791
[Release][Doc] add 3.1.0 release ( #12227 )
...
* [Release] add 3.1.0 release to doc
2 years ago
308f632fcc
[Feature][TASK-PLUGIN] AWS Database Migration Service (DMS) ( #11868 )
...
* add aws dms
2 years ago
d3f087e29b
[feat][ci] Add pre-commit for easy spotless config ( #12192 )
...
* Use pre-commit to handle spotless check
* Migrate exists python pre-commit-config into dir root one
* Change related document
2 years ago
995c1480f9
[Document][Improvement] Polish docs for standalone deployment ( #12181 )
...
* Polish docs for standalone deployment
2 years ago
4973b30efb
[Feature][doc] Add a guide to nominate new committer ( #12175 )
...
Co-authored-by: Kirs <ACM_China@yeah.net>
2 years ago
b95db44130
[fix][doc] Do not change docsdev.js during releasing ( #12151 )
...
ref: https://github.com/apache/dolphinscheduler/pull/12146#discussion_r979708629
and https://github.com/apache/dolphinscheduler-website/pull/829
2 years ago
4269fa7ea7
[release][doc] Add release 3.0.1 ( #12139 )
...
* add release 3.0.1
* add 30x version in the github bug report
2 years ago
779a0b9c75
[Improvement-#11608][task-plugin] New time variables are added to facilitate business development ( #11667 )
...
Add new time expressions for users' convenience
Co-authored-by: 冯剑 Jian <jian.feng@jiduauto.com>
Co-authored-by: David <dailidong66@gmail.com>
2 years ago
e6832220c3
[Improvement][SQL Task]use default sql segment separator ( #10869 )
2 years ago
064224696f
[Doc][Improvement] Add instructions for process execution type and data complement ( #12121 )
...
* Add instructions for process execution type and data complement
* Fix and polish the docs
2 years ago
08a4c7981f
[improve] Remove the spark version of spark task ( #11860 )
2 years ago
cf522e2fc0
[improve] Optimize MLFlow task plugin for easy of use ( #12071 )
...
optimize code
2 years ago
66958b91d7
[Feature][Resource Center] Add support for Alibaba Cloud OSS as storage of resource center ( #11708 )
...
* Add support for Alibaba Cloud OSS as storage of resource center
* Add dependency licenses
* Refactor code, add unit tests and related docs
2 years ago
8cddb10773
[Feat][UI] Update package. ( #12046 )
...
* [Feat][UI] Update package.
* [Feat][UI] Update package.
2 years ago
89d55bc9b7
Fix link errors in release documentation. ( #12049 )
2 years ago
a44817fc46
[Doc][Style] Fix doc format once for all ( #12006 )
2 years ago
5b6bebf7d5
[fix][doc] Update the registry related configuration in values.yaml ( #11444 )
2 years ago
86291bb5e6
[Feature-10273][Datasource] Add dataSource for AWS Athena ( #11157 )
2 years ago
b73a9511a7
Remove sensitive configs from dolphinscheduler_env.sh ( #11917 )
2 years ago
a88cb02ee8
[improve][Doc] Put default task parameters in a new file ( #11776 )
...
* Put default task parameters in a new file
* Add links to task plugin default parameters appendix docs
* Update Chinese docs
2 years ago
0deafba12c
fix procedure task param pass ( #11919 )
2 years ago
892d867270
[Doc][Task Plugin] Fix MLflow task plugin doc ( #11905 )
...
* update mlflow doc
* fix punctuations
2 years ago
Aaron Wang
Wenjun Ruan
Yiming Guo
Kerwin
houshitao
JieguangZhou
Jiajie Zhong
Eric Gao
simsicon
Yann Ann
limaiwang
Chris-Arith
tuchg
fuchanghai
Tq
TyrantLucifer
David
Yann Ann
caishunfeng
darkz
Jiajie Zhong
冯剑
zhuxt2015
rickchengx
songjianet
ZhaoGuodong
kezhenxu94