gabry.wu
0e5fb86bef
Adapting partial code(file name start with N) to the sonar cloud rule ( #2215 )
5 years ago
break60
e388e343db
dd branch flow node verification
5 years ago
break60
b90461fae9
Merge branch 'dev' of https://github.com/apache/incubator-dolphinscheduler into dev
5 years ago
xingchun-chen
7789e05c20
add LoginTest license ( #2217 )
...
Co-authored-by: chenxingchun <438044805@qq.com>
5 years ago
break60
99c28c7485
modify parameters ( #2203 )
...
* Delete worker group management page
* Modify workgroupid parameter name
* Worker group parameter modification
* modify parameters
5 years ago
qiaozhanwei
055bb28de4
worker fault tolerance modify ( #2212 )
...
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* add license
* modify javadoc error
* TaskExecutionContext create modify
* buildAckCommand taskInstanceId not set modify
* java doc error modify
* add comment
* ExecutorManager interface add generic type
* add TaskInstanceCacheManager receive Worker report result
* TaskInstance setExecutePath
* add TaskInstanceCacheManager to receive Worker Task result report
* TaskInstanceCacheManager add remove method
* add license
* add dispatcht task method
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* 1,worker TaskPros use TaskExecutionContext replase
2,Master kill Task , KillTaskProcessor modify
* worker remove db
* ShellTask modify
* master persistence processId and appIds
* master persistence processId and appIds
* master add kill task logic
* master add kill task logic
* master add kill task logic
* javadoc error modify
* remove chinese log
* executeDirectly method add Override
* remote module modify
* TaskKillResponseProcessor command type modify
* create buildKillCommand
* host add host:port format
* host add host:port format
* TaskAckProcessor modify
* TaskAckProcessor modify
* task prioriry refator
* remove ITaskQueue
* task prioriry refator
* remove ITaskQueue
* TaskPriority refactor
* remove logs
* WorkerServer refactor
* MasterSchedulerService modify
* WorkerConfig listen port modify
* modify master and worker listen port
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* Encapsulate the parameters required by sqltask
* 1,Encapsulate the parameters required by sqltask
2,SQLTask optimization
* AbstractTask modify
* ProcedureTask optimization
* MasterSchedulerService modify
* TaskUpdateQueueConsumer modify
* test
* DataxTask process run debug
* DataxTask process run debug
* add protobuf dependency,MR、Spark task etc need this
* TaskUpdateQueueConsumer modify
* TaskExecutionContextBuilder set TaskInstance workgroup
* WorkerGroupService queryAllGroup modify
query available work group
* 1,get workergroup from zk modify
2,SpringConnectionFactory repeat load modify
* master and worker register ip use OSUtils.getHost()
* ProcessInstance host set ip:port format
* worker fault tolerance modify
* Constants and .env modify
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
5 years ago
xingchun-chen
c82758c505
e2e add the case of deleting user and tenant ( #2210 )
...
* add e2e
5 years ago
qiaozhanwei
cc878505cd
1,master and worker register ip use OSUtils.getHost() 2,ProcessInstance host set ip:port format ( #2209 )
...
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* add license
* modify javadoc error
* TaskExecutionContext create modify
* buildAckCommand taskInstanceId not set modify
* java doc error modify
* add comment
* ExecutorManager interface add generic type
* add TaskInstanceCacheManager receive Worker report result
* TaskInstance setExecutePath
* add TaskInstanceCacheManager to receive Worker Task result report
* TaskInstanceCacheManager add remove method
* add license
* add dispatcht task method
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* 1,worker TaskPros use TaskExecutionContext replase
2,Master kill Task , KillTaskProcessor modify
* worker remove db
* ShellTask modify
* master persistence processId and appIds
* master persistence processId and appIds
* master add kill task logic
* master add kill task logic
* master add kill task logic
* javadoc error modify
* remove chinese log
* executeDirectly method add Override
* remote module modify
* TaskKillResponseProcessor command type modify
* create buildKillCommand
* host add host:port format
* host add host:port format
* TaskAckProcessor modify
* TaskAckProcessor modify
* task prioriry refator
* remove ITaskQueue
* task prioriry refator
* remove ITaskQueue
* TaskPriority refactor
* remove logs
* WorkerServer refactor
* MasterSchedulerService modify
* WorkerConfig listen port modify
* modify master and worker listen port
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* Encapsulate the parameters required by sqltask
* 1,Encapsulate the parameters required by sqltask
2,SQLTask optimization
* AbstractTask modify
* ProcedureTask optimization
* MasterSchedulerService modify
* TaskUpdateQueueConsumer modify
* test
* DataxTask process run debug
* DataxTask process run debug
* add protobuf dependency,MR、Spark task etc need this
* TaskUpdateQueueConsumer modify
* TaskExecutionContextBuilder set TaskInstance workgroup
* WorkerGroupService queryAllGroup modify
query available work group
* 1,get workergroup from zk modify
2,SpringConnectionFactory repeat load modify
* master and worker register ip use OSUtils.getHost()
* ProcessInstance host set ip:port format
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
5 years ago
t1mon
7dd126da71
Optimize PropertyUtils instantiation. ( #2199 )
...
* Optimize PropertyUtils instantiation.
* Fix info error.
5 years ago
gabry.wu
c5e1c55c00
Adapting partial code(file name start with M #2 ) to the sonar cloud rule ( #2179 )
...
* Adapting partial code(file name start with M #2 ) to the sonar cloud rule
* remove unused import
5 years ago
qiaozhanwei
e31b48fdb2
1,get workergroup from zk modify 2,SpringConnectionFactory repeat load modify ( #2201 )
...
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* add license
* modify javadoc error
* TaskExecutionContext create modify
* buildAckCommand taskInstanceId not set modify
* java doc error modify
* add comment
* ExecutorManager interface add generic type
* add TaskInstanceCacheManager receive Worker report result
* TaskInstance setExecutePath
* add TaskInstanceCacheManager to receive Worker Task result report
* TaskInstanceCacheManager add remove method
* add license
* add dispatcht task method
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* 1,worker TaskPros use TaskExecutionContext replase
2,Master kill Task , KillTaskProcessor modify
* worker remove db
* ShellTask modify
* master persistence processId and appIds
* master persistence processId and appIds
* master add kill task logic
* master add kill task logic
* master add kill task logic
* javadoc error modify
* remove chinese log
* executeDirectly method add Override
* remote module modify
* TaskKillResponseProcessor command type modify
* create buildKillCommand
* host add host:port format
* host add host:port format
* TaskAckProcessor modify
* TaskAckProcessor modify
* task prioriry refator
* remove ITaskQueue
* task prioriry refator
* remove ITaskQueue
* TaskPriority refactor
* remove logs
* WorkerServer refactor
* MasterSchedulerService modify
* WorkerConfig listen port modify
* modify master and worker listen port
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* Encapsulate the parameters required by sqltask
* 1,Encapsulate the parameters required by sqltask
2,SQLTask optimization
* AbstractTask modify
* ProcedureTask optimization
* MasterSchedulerService modify
* TaskUpdateQueueConsumer modify
* test
* DataxTask process run debug
* DataxTask process run debug
* add protobuf dependency,MR、Spark task etc need this
* TaskUpdateQueueConsumer modify
* TaskExecutionContextBuilder set TaskInstance workgroup
* WorkerGroupService queryAllGroup modify
query available work group
* 1,get workergroup from zk modify
2,SpringConnectionFactory repeat load modify
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
5 years ago
Rubik-W
69633a29c6
fix: Api server startup fail ( #2198 )
5 years ago
tswstarplanet
f28ba55014
fix ResultSet not close and reformat code ( #2183 )
...
* fix ResultSet not close
* fix ResultSet not close
* extract code of closing resource to a method
* remove redundant if condition
* modify e2e timeout
5 years ago
tswstarplanet
a3767642a5
change static field to instance field ( #2190 )
5 years ago
t1mon
f6f29f4abb
Replace Charset.forName("UTF-8") with StandardCharsets.UTF_8. ( #2193 )
5 years ago
Tboy
11dde20311
Refactor worker ( #2189 )
...
* let quartz use the same datasource
* move master/worker config from dao.properties to each config
add master/worker registry test
* move mybatis config from application.properties to SpringConnectionFactory
* move mybatis-plus config from application.properties to SpringConnectionFactory
5 years ago
break60
aa650b4426
Worker group parameter modification ( #2188 )
...
* Delete worker group management page
* Modify workgroupid parameter name
* Worker group parameter modification
5 years ago
gabry.wu
1658176ef5
Adapting partial code(file name start with P #1 ) to the sonar cloud rule ( #2171 ) ( #2187 )
5 years ago
tswstarplanet
e0fc3171da
Cache HadoopUtils instance with specific days expire time ( #2181 )
...
* Cache HadoopUtils instance with 7 days expire time
* solve sonar issue
* add kerberos expire time config
* move KERBEROS_EXPIRE_TIME to Constants.java
5 years ago
gabry.wu
fcc57320d6
Adapting partial code(file name start with M #1 ) to the sonar cloud rule ( #2173 )
...
* Adapting partial code(file name start with M #1 ) to the sonar cloud rule
* remove unused import
5 years ago
Tboy
75efbb5402
Refactor worker ( #2186 )
...
* let quartz use the same datasource
* move master/worker config from dao.properties to each config
add master/worker registry test
5 years ago
gabry.wu
61ebbf2946
Adapting partial code(file name start with L) to the sonar cloud rule ( #2171 )
5 years ago
Rubik-W
e0389aebf6
fix: NullPointException in the case of task queuing ( #2175 )
5 years ago
gabry.wu
a076ae4c36
Adapting partial code(file name start with J) to the sonar cloud rule ( #2165 )
...
* Adapting partial code(file name start with J) to the sonar cloud rule
* fix unit failure
5 years ago
Rubik-W
80c6ce5711
The master and worker server exit exception #2163 ( #2176 )
...
* fix : #2163
* fix: format
5 years ago
DS
2e63dc9c96
Delete .asf.yaml ( #2168 )
5 years ago
DS
e98c6f3008
Update .asf.yaml ( #2167 )
5 years ago
DS
205af16d55
Create .asf.yaml ( #2166 )
5 years ago
gabry.wu
daad5efbab
use static method in static class JSON ( #2156 )
5 years ago
liwenhe1993
dd2c50a10d
Toolbar of the DAG has not prompt ( #2161 )
...
* Toolbar of the DAG has not prompt
* Remove duplication code
5 years ago
qiaozhanwei
71b11e2c08
DataxTask process test modify ( #2162 )
...
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* add license
* modify javadoc error
* TaskExecutionContext create modify
* buildAckCommand taskInstanceId not set modify
* java doc error modify
* add comment
* ExecutorManager interface add generic type
* add TaskInstanceCacheManager receive Worker report result
* TaskInstance setExecutePath
* add TaskInstanceCacheManager to receive Worker Task result report
* TaskInstanceCacheManager add remove method
* add license
* add dispatcht task method
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* 1,worker TaskPros use TaskExecutionContext replase
2,Master kill Task , KillTaskProcessor modify
* worker remove db
* ShellTask modify
* master persistence processId and appIds
* master persistence processId and appIds
* master add kill task logic
* master add kill task logic
* master add kill task logic
* javadoc error modify
* remove chinese log
* executeDirectly method add Override
* remote module modify
* TaskKillResponseProcessor command type modify
* create buildKillCommand
* host add host:port format
* host add host:port format
* TaskAckProcessor modify
* TaskAckProcessor modify
* task prioriry refator
* remove ITaskQueue
* task prioriry refator
* remove ITaskQueue
* TaskPriority refactor
* remove logs
* WorkerServer refactor
* MasterSchedulerService modify
* WorkerConfig listen port modify
* modify master and worker listen port
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* Encapsulate the parameters required by sqltask
* 1,Encapsulate the parameters required by sqltask
2,SQLTask optimization
* AbstractTask modify
* ProcedureTask optimization
* MasterSchedulerService modify
* TaskUpdateQueueConsumer modify
* test
* DataxTask process run debug
* DataxTask process run debug
* add protobuf dependency,MR、Spark task etc need this
* TaskUpdateQueueConsumer modify
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
5 years ago
gaojun2048
a0088c1137
fix rpmbuild and DS Ambari plugin bug ( #2144 )
...
* Update quick-start.md
* 更新demo的地址
* add rpm plugin
* delete the git ignore files
* 测试git ignore
* 测试ignore
* optimize the .gitignore file
* add rpm package, delete unuse copy jar action in assembly
* delete docs dir
* add rpm plugin support
* add rpm plugin support
* add rpm plugin support
* dolphinscheduler-common have not bin dir
* dolphinscheduler-common have not bin dir
* delete unuse config about assembly
* add defineSatement
* add rpm plugin support
* add rpm plugin support
* add rpm plugin support
* add rpm support
* update install dirname , make front and backend together
* update rpm name
* update rpm name
* update rpm config
* add jars excludes
* add jars excludes
* add rpm plugion
* add rpm plugion
* add rpm plugion
* add auto create /opt/soft
* The install dir can not be fixed to apache-dolphinscheduler-incubating-1.2.1 , its version is 1.2.2
* install from rpm will create a link
* the ui install thrown rpm only support jetty not nginx
* the ui install thrown rpm only support jetty not nginx
* sql目录里加上soft_version文件
* env dir need in conf dir
* remove no use module
* add module
* updaate
* rpmbuild add dolphinscheduler-service module
5 years ago
gabry.wu
93aa160698
Adapting partial code(file name start with I) to the sonar cloud rule ( #2155 )
5 years ago
qiaozhanwei
607ec3d174
ProcedureTask process test modify ( #2159 )
...
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* 1, master persistent task
2. extract master and worker communication model
* add license
* modify javadoc error
* TaskExecutionContext create modify
* buildAckCommand taskInstanceId not set modify
* java doc error modify
* add comment
* ExecutorManager interface add generic type
* add TaskInstanceCacheManager receive Worker report result
* TaskInstance setExecutePath
* add TaskInstanceCacheManager to receive Worker Task result report
* TaskInstanceCacheManager add remove method
* add license
* add dispatcht task method
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* AbstractCommandExecutor remove db access
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* taskInstanceCache is null ,need load from db
* 1,worker TaskPros use TaskExecutionContext replase
2,Master kill Task , KillTaskProcessor modify
* worker remove db
* ShellTask modify
* master persistence processId and appIds
* master persistence processId and appIds
* master add kill task logic
* master add kill task logic
* master add kill task logic
* javadoc error modify
* remove chinese log
* executeDirectly method add Override
* remote module modify
* TaskKillResponseProcessor command type modify
* create buildKillCommand
* host add host:port format
* host add host:port format
* TaskAckProcessor modify
* TaskAckProcessor modify
* task prioriry refator
* remove ITaskQueue
* task prioriry refator
* remove ITaskQueue
* TaskPriority refactor
* remove logs
* WorkerServer refactor
* MasterSchedulerService modify
* WorkerConfig listen port modify
* modify master and worker listen port
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* cancelTaskInstance set TaskExecutionContext host,logPath,executePath
* Encapsulate the parameters required by sqltask
* 1,Encapsulate the parameters required by sqltask
2,SQLTask optimization
* AbstractTask modify
* ProcedureTask optimization
* MasterSchedulerService modify
* TaskUpdateQueueConsumer modify
* test
Co-authored-by: qiaozhanwei <qiaozhanwei@analysys.com.cn>
5 years ago
dailidong
455d8c7a7c
Delete .asf.yaml ( #2158 )
5 years ago
Yelli
91ecc549a1
fix bug: data too long for column 'app_link' ( #2146 )
...
* fix bug: data to long for app_link field
* change app_link field length in 1.2.2 ddl
* change app_link field type to text
5 years ago
zhukai
68e4e5c4e0
add input cache for sqoop and conditions tasks ( #2118 )
...
Co-authored-by: zhukai <zhukai@ygsoft.com>
5 years ago
gabry.wu
7473d5e928
Adapting partial code(file name start with H) to the sonar cloud rule ( #2145 )
5 years ago
Rubik-W
9fda4da70e
fix: The toolbar buttons of Dag disable bug ( #2124 )
5 years ago
break60
712a1bc126
Limit customization file content to no more than 3000 lines( #2128 ) ( #2141 )
...
* Change DOM label
* Change name to lowercase
* Limit customization file content to no more than 3000 lines
5 years ago
t1mon
46e30b1e50
Add mail server configuration description information ( #2143 )
...
* Add mail server configuration description information
* Remove mail server configuration note in alert.properties
5 years ago
t1mon
16eeaf50b4
Fix javadoc and DataxTaskTest UT error ( #2147 )
...
* Fix javadoc and DataxTaskTest UT error
* Fix UT error
* LoggerServiceTest UT error
* WorkerGroupServiceTest UT error
Modify Host.of()
* Add address NPE check
* Trigger actions rerun
* Try to solve maven VM crash
* Try to solve maven VM crash 1
* Try to solve maven VM crash 2
* Try to solve maven VM crash 3
5 years ago
tswstarplanet
afb907b4b0
fix unequal number of arguments and placeholder ( #2149 )
5 years ago
tswstarplanet
e3065dbfcc
fix field access modifier ( #2150 )
5 years ago
tswstarplanet
e27ba3fd97
use logger to print exception ( #2151 )
5 years ago
liwenhe1993
ebfacbfcfa
docker run failed when docker command was `docker run -d` ( #2125 )
...
* docker run failed when docker command was `docker run -d`
1. add tini
2. replace `tee` command by `tail` command in startup.sh
3. api-server need to support zookeeper
* modify `check.sh` in dockerfile
* add init directories and log files
5 years ago
Jave-Chen
b03e0a02c5
fix wrong sonar analysis with checkout V2. ( #2148 )
5 years ago
break60
84bcdc7896
Limit customization file content to no more than 3000 lines
5 years ago
AhahaGe
1db4ac6b63
edit README.md modify word from expand to extend ( #2138 )
...
Co-authored-by: guirong.ggr <guirong.ggr@dtwave-inc.com>
5 years ago
Tboy
7f72df4ca2
let quartz use the same datasource ( #2137 )
5 years ago