3026f04d85
[Improvement-5452][Task] ds flink task support submit a PyFlink job via the CLI ( #5453 )
...
* flink task support submit a PyFlink job via the CLI.
* optimize attribute name.
* Modify pyflink parameter judgment logic
3 years ago
86ffc1f3f2
[Bug][WorkerServer] SqlTask NullPointerException ( #5556 )
...
* [Bug][WorkerServer] add null checks to sqlParamsMap and sqlParamsMap.get(i), Add test to verify it
* Change Test Name
* Fix Code style issues, Modify checking null for Sql params in sqlParamsMap with clearer logging.
3 years ago
67711442d5
[Improvement][SQL] Query return number In SQL should be configurable ( #5632 )
...
* [Improvement][SQL] Query return number should be configurable #5580
* Update SqlTaskTest.java
* Update sql.vue
3 years ago
e2d6265e26
[Improvement][registry-plugin] Optimize registry plugin loading and initial installation #5635
3 years ago
3fb01e1b4c
[Fix-5596][Python] Fix conflict between python_home and datax_home configuration in dolphinscheduler_env.sh ( #5612 )
3 years ago
053e548bf3
[Improvement-5539][Master] Check status of taskInstance from cache ( #5572 )
...
* improvement:check status of taskInstance from cache
* issue5572 use timer instead of while&sleep; consider concurrent modification
* use computeifpresent instead of lock
* simplify getByTaskInstanceId function
* add ut for TaskInstanceCacheManagerImpl; fix bug in TaskInstanceCacheManagerImpl
* add Apache license header;add test class in root pom
3 years ago
30c88032d3
[Feature-#3961][Registry]Registry-SPI ( #5562 )
...
[Feature#3961]Registry SPI
All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
3 years ago
3193837f5a
[DS-5559][fix][Master Server] Master Server was shutdown but the process still in system ( #5588 )
...
* Close spring context to destory beans that has running thread
3 years ago
8bf042ae6e
[Fix-5596][Python] Fix conflict between python_home and datax_home configuration in dolphinscheduler_env.sh ( #5612 )
4 years ago
79eb2e85d7
[Improvement-5539][Master] Check status of taskInstance from cache ( #5572 )
...
* improvement:check status of taskInstance from cache
* issue5572 use timer instead of while&sleep; consider concurrent modification
* use computeifpresent instead of lock
* simplify getByTaskInstanceId function
* add ut for TaskInstanceCacheManagerImpl; fix bug in TaskInstanceCacheManagerImpl
* add Apache license header;add test class in root pom
4 years ago
e2243d63be
[Feature-#3961][Registry]Registry-SPI ( #5562 )
...
[Feature#3961]Registry SPI
All the logical structure of the registry must be converted into a tree structure within the system, so some plug-ins must be converted internally, such as ETCD
The registry supports distributed locks.
todo: The specific information about the registration center of the API module needs to be adjusted.
4 years ago
75be09735a
[DS-5559][fix][Master Server] Master Server was shutdown but the process still in system ( #5588 )
...
* Close spring context to destory beans that has running thread
4 years ago
a5a0c7c5f8
[Fix-5570][Worker] Fix worker.groups in worker.properties is still commented and improve install scripts and configurations ( #5571 )
4 years ago
56273319a2
[Bug][Master] remove check with executePath when kill yarn job ( #5552 )
...
* [Bug][Master] remove check with executePath when kill yarn job
* [Bug][Master] init execute path if not exist
* [Bug][Master] move createWorkDirIfAbsent out side of executePath calculate part
Co-authored-by: wanggang <wanggy01@servyou.com.cn>
4 years ago
842c5400e6
[Improvement][Server] Must restart master if Zk reconnect ( #5210 ) ( #5211 )
4 years ago
9d0c816cee
[Bug][worker] task throw ConcurrentModifiedException #5528 ( #5530 )
...
* [Bug][worker] task throw ConcurrentModifiedException #5528
* fix code smell
4 years ago
b537f4eb26
[Improvement][Docker/K8s] Support more configurations and service accesses, reduce the duplication configurations, improve image for python and the compatibility of python home ( #5393 )
...
* [Improvement][Docker] Support more configurations and improve image for python
* [Improvement][K8s] Support more configurations, more service accesses, and reduce the duplication configurations
* [Improvement][Python] Improve the compatibility of python home
* [Improvement][Install] Fix install config and install.sh
* [Improvement][Install] Fix workflow ut
* [Improvement][Docker] Optimize docker-swarm/check script
* [Improvement][DB] Update default username and password of database
* Update comments in master/worker.properties
* Specify version for all images
* [Improvement][Docker] Optimize PS1 and WORKDIR
* [Improvement][SQL] Reduce heap size to 64m in create-dolphinscheduler.sh and upgrade-dolphinscheduler.sh
* [Fix-5431][K8s] Fix master and worker cannot get the right address with custom DNS
4 years ago
1b7dcea610
[Improvement-5440][Maven] Project module maven dependency optimization ( #5441 )
...
* Optimize dependencies between modules.
* update module-alert dependency.
* add repository in the pom xml.
* update Central Repository in the pom.xml.
4 years ago
bc22ae7c91
[Improvement][Task] Remove deprecated TaskRecordDao and simply the after() method in the AbstractTask class ( #5492 )
...
* remove the depreated data quality module
* fix lint
Co-authored-by: Yao <ywang46@paypal.com>
4 years ago
dd80f98196
update sleep exception
4 years ago
7aa8b518ac
Merge remote-tracking branch 'upstream/dev' into json_split
...
# Conflicts:
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
4 years ago
2dd0d936fb
Merge remote-tracking branch 'upstream/dev' into json_split
...
# Conflicts:
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/runner/MasterExecThread.java
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/utils/DependentExecute.java
4 years ago
515fa4912e
[Feature][JsonSplit] Fix ProcessService ut ( #5435 )
...
* Fix ProcessService ut
* codeStyle
* Fix AlertPlugin ut
* Fix AlertPlugin ut
* Fix server module ut
* Fix server module ut
* Fix api ut
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
9c77faa8ac
[Improvement-5058][Master] The master-server creates the DB data source twice ( #5139 )
...
* reduce the number of data sources created twice by master server.
* update ProcessAlertManagerTest test class to increase coverage.
* add DolphinSchedulerPluginLoaderTest test ignore.
Co-authored-by: zhuang chong
4 years ago
c850b85484
Merge remote-tracking branch 'upstream/dev' into json_split
...
# Conflicts:
# dolphinscheduler-api/src/main/java/org/apache/dolphinscheduler/api/controller/ProcessDefinitionController.java
# dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/task/procedure/ProcedureTask.java
4 years ago
88a07f7b55
[Feature-4093][server] Support for stored procedures and stored function calls and data source supports DB2 ( #4094 )
...
* Support for stored procedures and stored function calls and data source supports DB2.
Co-authored-by: zhuangchong <zhuangchong8@163.com>
4 years ago
c3558965c1
update taskParams/add task delayTime/fix conditionType bug ( #5385 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
e6d8da484f
[Improvement][API] Create Datasource #5083 ( #5116 )
...
* [Improvement][API] Create Datasource #5083
* Refact create/ update datasource API
* fix Code Smells
* fix code smell
* resolve code smell
* use ConnectionParam to replace BaseDatasource
* solve license
* add ut
* code check
* add ut
* fix ut coverage
* fix ut
4 years ago
75d1ad645c
fix bug: 5349, work flow would running if task is killed by manual. ( #5362 )
...
Co-authored-by: baoliang <baoliang@analysys.com.cn>
4 years ago
862565a7c7
[Feature][JsonSplit] Fix dependTask bug ( #5360 )
...
* update SnowFlake
* update processDefinite from processInstance
* update processDefinite from processInstance
* Fix task logger path
* Fix dependTask bug
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
abdd2337b1
[Bug][worker] Fix sql task query result is empty ( #5322 ) ( #5324 )
...
* [Bug][worker] Fix sql task query result is empty (#5322 )
* add ut
* remove duplicate
* add ut
4 years ago
e57ee658e8
[Improvement] Optimized close of IO ( #5336 )
...
* [improvement]modify all methods that refer to LogClientService in the code.
* delete unused code
* [Improvement] Optimized close of IO
4 years ago
feffd972b5
[Feature][JsonSplit] Fix task logger path ( #5331 )
...
* update SnowFlake
* update processDefinite from processInstance
* update processDefinite from processInstance
* Fix task logger path
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
49b234609a
update SnowFlake ( #5321 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
513eb76919
[Improvement-5147][Service]This judgment always returns true ( #5156 )
...
* this expression which always evaluates "true"
* delete unused variables
* LogClientService add AutoCloseable
* reformat code
* reformat code
* reformat code
* add log in ProcessService
* reformat code
* Optimize the code
* Optimize the code
* [improvement]modify all methods that refer to LogClientService in the code.
* delete unused code
4 years ago
83519bdfc3
[Improvement][Server] Optimize log display ( #5311 )
4 years ago
f7a06a033d
[Feature][JsonSplit] Fix master/worker about shellTask ( #5300 )
...
* Fix quoted bug about processDefineId of processInstance
* Fix master/worker about shellTask
* codeStyle
* codeStyle
* update taskType
* codeStyle
* update timeoutNotifyStrategy
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
e39409a412
[Improvement-5275][Server] Fix wrong excludeFilters and move zk/ZKMasterClient and registry/ServerNodeManager into master ( #5278 )
...
* [Improvement-5275][Server] Move zk/ZKMasterClient and registry/ServerNodeManager into master
* [Improvement-5275][Server] Fix code smells
* [Improvement-5275][Server] Add ZKMasterClientTest
4 years ago
1c1f6663a6
[Improvement][WorkerGroup] Optimize and reduce database and zookeeper queries and replace worker address input with multiple-select ( #5182 )
...
* [Improvement][Server] Optimize and reduce database and ZooKeeper query
* [Improvement][WorkerGroup] Replace worker address input with multiple-select
* [Improvement][WorkerGroup] Fix unit test
* [Improvement][API&UI] Optimize long host to short host and long host display in k8s
4 years ago
cc7a4446f5
[Improvement][Task] DataX Node doesn't Support Clickhouse as datasource ( #5180 ) ( #5243 )
...
* [Improvement][Module Name] DataX Node doesn't Support Clickhouse as the datasource (#5180 )
* add ut
* add ut
* resolve code smell
4 years ago
645847c096
Fix quoted bug about processDefineId of processInstance ( #5263 )
...
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
73f5a8e0a7
[Feature][JsonSplit] Fix master/worker server ( #5253 )
...
* Fix dependency failure
* Fix taskInstance
* task instance list page
* code review
* fix logger path
* Fix master/worker server
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
31188ffca0
[Feature][JsonSplit] Fix taskInstance ( #5246 )
...
* Fix dependency failure
* Fix taskInstance
* task instance list page
* code review
* fix logger path
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
4a1303b625
[Improvement][Worker] Kill task error( #5199 ) ( #5212 )
4 years ago
a8f2894b79
[Feature][JsonSplit] Fix master/processInstance bug ( #5206 )
...
* transform taskCode from long to string
* fix process bug
* code review
* code review
* code review
* Fix master/processInstance bug
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
86ce8f133a
[JsonSplit-4417][master/worker/api]json split- remove json in task instance. ( #5160 )
...
* [json split] refactor json in task instance
* add task_pramas in task_instance
* feature #4417 refactor json split in task instance
* code style
* code style
* code style
* code style
* code style
* update
* update
* update unit test
* update
4 years ago
97fb08f0ce
[Improvement][LogServer] Thread pool parameter adjustment #5161 ( #5163 )
...
* [Improvement][LogServer] Thread pool parameter adjustment #5161
* change log request processor thread num 2 * cpuCores + 1
4 years ago
dd36da8d1f
[Feature][JsonSplit] fix process bug ( #5193 )
...
* transform taskCode from long to string
* fix process bug
* code review
* code review
* code review
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
e53369318b
[Fix-4721][worker]The shell background starts the YARN task scenario, and the kill function is abnormal ( #4722 )
...
* fix the shell starts the yarn task in the background
* update StringUtils code style.
* solve code smell.
* add method comment in StringUtils class.
* update AlertGroupMapper code style.
* update AlertGroupMapperTest
* update sql script test.
4 years ago
a467fffd7b
[Feature][JsonSplit] fix taskId in json ( #5165 )
...
* modify checkDAGRing and ProcessService method
* merge
* modify dagRing
* modify process instance for project home page
* fix save process bug
* codeStyle
* Fix logical bug in saving process definition
* codeSytle
* Fix bug in interface of queryProcessDefinitionList
* codeSytle
* Fix api bug"
* fix taskId in processDefinitionJson
* fix json taskId
* codeStyle
Co-authored-by: JinyLeeChina <297062848@qq.com>
4 years ago
zhuangchong
LOUKHNATI Mohamed Khalil
kyoty
Shiwen Cheng
blackberrier
Kirs
Chouc
geosmart
ruanwenjun
Yao WANG
baoliang
JinyLeeChina
bao liang
luoyuan