DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
Wenjun Ruan	db595b3eff	Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516 ) * Optimize master log, add workflow instance id and task instance id in log * Use MDC to set the workflow info in log4j * Add workflowInstanceId and taskInstanceId in MDC	3 years ago
WangJPLeo	f46faa02c3	[Fix-10452] Serial wait for policy recovery (#10453 ) * Serial wait for policy recovery * processInstance state check null * add sendresult(Host host, Command command) method and replace the original.	3 years ago
Kerwin	cefcb6d556	[Fix-10522] [API] Fix get executable time null pointer exception (#10523 )	3 years ago
Eric Gao	cc06eaaf54	[Improvement][Metrics] Apply micrometer naming convention to metrics (#10477 ) * Apply micrometer naming convention to worker metrics * Apply micrometer naming convention all current metrics * Fix remaining metrics names, update English docs and add Chinese docs * Fix metrics names in grafana-demo dashboards	3 years ago
Wenjun Ruan	0bdfa0cff9	Fix PeerTaskInstancePriorityQueue cannot contains method use taskInstanceId to check (#10491 )	3 years ago
Wenjun Ruan	ad2646ff1f	Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479 ) * Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe	3 years ago
hstdream	f6fea06f10	[Improve] Enhance complement function transformation (#10376 )	3 years ago
Wenjun Ruan	b0d9d3f9ab	Add some warning log in master (#10383 ) * Add some warn log in master * fix may skip sleep	3 years ago
Wenjun Ruan	e21d7b1551	[Feature][metrics] Add master, worker metrics (#10326 ) * Add mater metrics * fix UT * Add url to mysql profile * Add worker metrics * Update grafana config * Add system metrics doc * Add process failover counter * Add metrics image * Change jpg to png * Add command insert metrics * Fix UT * Revert UT	3 years ago
Wenjun Ruan	3ecbee3885	Optimize MasterServer, add MasterRPCService (#10371 ) * Optimize MasterServer, avoid NPE	3 years ago
xiangzihao	b1fb17119b	[Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373 )	3 years ago
Wenjun Ruan	2d3be6b36c	Add dolphinscheduler-scheduler module (#10360 ) * Add dolphinscheduler-scheduler module	3 years ago
Wenjun Ruan	022e4886be	Remove quartz at WorkerServer (#10358 ) * Remove quartz at WorkerServer * move k8s and permission from dolphinscheduler-service to dolphinscheduler-api	3 years ago
Wenjun Ruan	0bf4043d5f	Move datasource params to datasouce plugin module (#10353 ) * Move datasource params to datasouce plugin module	3 years ago
WangJPLeo	e3e39cbdea	[Fix] Privilege Reconstruction Data Query Fix & Resource Creation Post Oper… (#10313 ) * Privilege Reconstruction Data Query Fix & Resource Creation Post Operations. * Resource Retry * e2e retry * create project log add * project list query log add * clean project log * delete delay * delete delay * remove post handle * project e2e * browser refresh * browser refresh * e2e fix * e2e browser refresh * rowk flow e2e fix * mapper deduplication * udf e2e * e2e	3 years ago
kezhenxu94	d80cf21456	Clean up unused dependencies and packaging issues (#9944 )	3 years ago
WangJPLeo	a4948f58e6	[Feature][Permission] Reconstruction of permissions of resource center and monitoring center. (#10307 ) * Reconstruction of permissions of resource center and monitoring center. * clear local logs. * resource query fix	3 years ago
hstdream	53ab6f7b92	[Feature][API] Modify the permissions of project management, security center, data source center and data quality module. (#10289 ) * Resource userName is null * [Fix-10082]Resource userName is null and udf function manager add a username * [Fix-10082]Resource userName is null and udf function manager add a username * [Fix-10082]Resource userName is null and udf function manager add a username * [Feature][API] Modify the permissions of project management, security center, data source center and data quality module. * [Feature][API] Modify the permissions of project management, security center, data source center and data quality module. * [Feature][API] Modify the permissions of project management, security center, data source center and data quality module. Co-authored-by: houshitao <shitaohou@163.com>	3 years ago
cadl	13af2adfc1	[fix] Fix the SERIAL_DISCARD execution_type not working (#10258 ) * fix always stop process_instance when execution_type is SERIAL_DISCARD * test: no running_process, submit it	3 years ago
WangJPLeo	7ac8ea493c	[Feature][Permission] Project management module authority control refactoring, authority abstract interface setting. (#10261 ) * Create process definition with task group and task group priority cannot be save into db * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * [Feature][API] Permission control of project * The result of the assertion cannot be changed. The only difference is the judgment condition. * test recovery. * controller test fix. * ProjectE2ETest fix. * WebElement import. * Prioritize problem solving. * Judging the length of the set fix Co-authored-by: houshitao <shitaohou@163.com> Co-authored-by: hstdream <33045461+hstdream@users.noreply.github.com>	3 years ago
BaoLiang	ee2b855ced	[BUG][TaskGroup] Task group does not take effect (#10093 ) * fix 10092: Task group does not take effect * fix 10092: Task group does not take effect * fix 10092: Task group does not take effect	3 years ago
He Zhao	5bb1eb04fc	[Feature][plugin] Add k8s task in task plugin (#9425 ) * [Feature][plugin] Add k8s task in task plugin * [Feature][plugin] fix dos and code problems * [Feature][plugin] refactor some code based on sonar * [Feature][UI] front-end for k8s task plugin * [Feature][plugin] delete some front files * [Feature][plugin] update document * Update docs/docs/zh/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * [Feature][UI] front-end change from review comments * [Feature][UI] replace get namespace list api * [Feature][plugin] change file name * Add kubernetes to zh task list Co-authored-by: hezhao2 <hezhao2@cisco.com> Co-authored-by: William Tong <weitong@cisco.com> Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>	3 years ago
Tq	d4aeee16e5	[Bug] [MASTER-9811]fix cmd param to overwrite global param when executing complement (#9952 ) * fix cmd param to overwrite global param when executing complement * fix cmd param to overwrite global param when executing complement	3 years ago
WangJPLeo	fb0f96ed94	[Fix-9868] A task flow definition isolates the runs of different execution strategies by version numbers. (#9869 ) * The thread cache task flow definition should get the latest version. * Coverage on New Code * Coverage on New Code * Coverage on New Code * use an existing method. * Increase unit test coverage. * Task flow definitions enforce policy isolation.	3 years ago
caishunfeng	5d5125d507	[Bug-9829][Api] Fix schedule timezone (#9830 )	3 years ago
WangJPLeo	5c0be8a3d7	A task instance that normally queries the serial wait state. (#9777 ) Co-authored-by: WangJPLeo <wangjipeng@whaleops.com>	3 years ago
Tq	e6dade71bb	fix process instance global param not include to task instance when master executing (#9730 )	3 years ago
caishunfeng	5657cb9aec	[Bug-9719][Master] fix failover fail because task plugins has not been loaded (#9720 )	3 years ago
gaojun2048	ebc4253d50	[fix][Service] BusinessTime should format with schedule timezone (#9714 ) * BusinessTime should format with schedule timezone * fix test error * fix test error * fix test error	3 years ago
caishunfeng	88d2803fe1	fix task dispatch error overload resource pool of task group (#9667 )	3 years ago
caishunfeng	63638601b0	fix process pause and rerun (#9568 )	3 years ago
Tq	fb0139e959	[Feature] [ALERT-9406]enrich alert info when insert new row into DB (#9445 ) * enrich alert info when insert new row into DB * fix comments	3 years ago
caishunfeng	b285ccf930	[Future-9396]Support output parameters transfer from parent workflow to child work flow (#9410 ) * [Future-9396]Support output parameters transfer from parent workflow to child work flow * fix note	3 years ago
Eric Gao	fd6b43bc81	[Dev] Switch version in pom.xml to dev-SNAPSHOT (#9223 ) (#9299 )	3 years ago
gaojun2048	5ef3f9d668	[optimization] [Service] Optimization ProcessService and add ProcessService interface (#9370 )	3 years ago
caishunfeng	ddf1ff98fa	[Improvement-7697][Master/Worker] Change the task ack to runnning callback (#8719 ) * rebase dev * change task state to dispatch if dispatch success * update task host when dispatch * add dispatch task event * test * check tenant after enable auto create * handle dispatch state Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
wangxj3	fd76436d87	[Feature-#8318] [master-server] Sub process transfer parameters to father process (#8522 ) * load param from subprocess * unit test * modify code style Co-authored-by: wangxj <wangxj31>	3 years ago
guoshupei	e2c1cc0579	[Fix-9222] [master] Support for deploying multiple masters on one node (#9240 ) This closes #9222 Co-authored-by: guoshupei <guoshupei@lixiang.com>	3 years ago
JinYong Li	85e56b5e2a	fix 9243 (#9244 )	3 years ago
Tq	03a0c9f544	[Bug] [ALERT-9224] fix wechat alert functions: app and group chat (#9225 ) * fix wechat alert with app and group chat * add apache header and fix imports * delete wechat alert 'userSendMsg' property code * delete wechat alert 'userSendMsg' property code in Test * delete wechat alert 'userSendMsg' property code in factory Test * change `getSubWorkFlowParam` from private to protected	3 years ago
worry	d3251c9bcc	[Improvement-9227][master]implement use the slot to scan the database (#9228 ) when the master assigns tasks by slot,implement use the slot to scan the database. This closes #9227	3 years ago
nobolity	0e3cafec1d	[Feature-8612][RESOURCE] extend s3 to the storage of ds (#8637 ) * feat(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.extend the type of storage 3.add the s3utils to manager resource 4.automatic inject the storage in addition to your config * fix(resource manager): update the dependency * fix(resource manager): extend s3 to the storage of ds fix the constant of hadooputils * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * fix(resource manager): merge the unitTest: 1.TenantServiceImpl 2.ResourceServiceImpl 3.UserServiceImpl * fix(resource manager): extend s3 to the storage of ds merge the resourceServiceTest * fix(resource manager): test cancel the test method createTenant verifyTenant * fix(resource manager): merge the code follow the check-result of sonar * fix(resource manager): extend s3 to the storage of ds fit the spell question * fix(resource manager): extend s3 to the storage of ds revert the common.properties * fix(resource manager): extend s3 to the storage of ds update the storageConfig with None * fix(resource manager): extend s3 to the storage of ds fix the judge of resourceType * fix(resource manager): extend s3 to the storage of ds undo the compile-mysql * fix(resource manager): extend s3 to the storage of ds delete hadoop aws * fix(resource manager): extend s3 to the storage of ds update the know-dependencies to delete aws 1.7.4 update the e2e file-manager common.properties * fix(resource manager): extend s3 to the storage of ds update the aws-region * fix(resource manager): extend s3 to the storage of ds fix the storageconfig init * fix(resource manager): update e2e docker-compose update e2e docker-compose * fix(resource manager): extend s3 to the storage of ds revent the e2e common.proprites print the resource type in propertyUtil * fix(resource manager): extend s3 to the storage of ds 1.println the properties * fix(resource manager): println the s3 info * fix(resource manager): extend s3 to the storage of ds delete the info and upgrade the s3 info to e2e * fix(resource manager): extend s3 to the storage of ds add the bucket init * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * fix(resource manager): extend s3 to the storage of ds upgrade the s3 endpoint * fix(resource manager): withPathStyleAccessEnabled(true) * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * fix(resource manager): upgrade the s3client builder * fix(resource manager): correct the s3 point to s3client * fix(resource manager): update the constant BUCKET_NAME * fix(resource manager): e2e s3 endpoint -> s3:9000 * fix(resource manager): extend s3 to the storage of ds 1.fix some spell question 2.delete the import * * style(resource manager): add info to createBucket * style(resource manager): debug the log * ci(resource manager): test test s3 * ci(ci): add INSERT INTO dolphinscheduler.t_ds_tenant (id, tenant_code, description, queue_id, create_time, update_time) VALUES(1, 'root', NULL, 1, NULL, NULL); to h2.sql * fix(resource manager): update the h2 sql * fix(resource manager): solve to delete the tenant * style(resource manager): merge the style end delete the unuse s3 config * fix(resource manager): extend s3 to the storage of ds UPDATE the rename resources when s3 * fix(resource manager): extend s3 to the storage of ds 1.fix the code style of QuartzImpl * fix(resource manager): extend s3 to the storage of ds 1.impoort restore_type to CommonUtils * fix(resource manager): update the work thread * fix(resource manager): update the baseTaskProcessor * fix(resource manager): upgrade dolphinscheduler-standalone-server.xml * fix(resource manager): add user Info to dolphinscheduler_h2.sql * fix(resource manager): merge the resourceType to NONE * style(upgrade the log level to info): * fix(resource manager): sysnc the h2.sql * fix(resource manager): update the merge the user tenant * fix(resource manager): merge the resourcesServiceImpl * fix(resource manager): when the storage is s3 ,that the directory can't be renamed * fix(resource manager): in s3 ,the directory cannot be renamed * fix(resource manager): delete the deleteRenameDirectory in E2E * fix(resource manager): check the style and recoverd the test * fix(resource manager): delete the log.print(LoginUser)	3 years ago
gaojun2048	9996f7a4fa	update some method to protected (#9169 )	3 years ago
caishunfeng	a93033b620	[Bug-8790][Api]fix Duplicate key TaskDefinition (#8986 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Kerwin	a1bef43981	[Feature-8579][Task] Task back-end plugin optimization, new plugins only need to modify the plugin's own module (#8584 ) * Task plug-in optimization, task-related classes are migrated to the task plugin module * remove tasktype enum class. * fix code style. * fix code style * fix code style * fix ProcessServiceTest.testUpdateResourceInfo test class * fix code style * fix test task plugin manager check parameters null * fix test ut * update aws license * change TaskTimeoutStrategy class	3 years ago
Kerwin	c4fc59c9bb	[Fix][Master] Fix master-server execution logic thread unsafe problem (#8707 ) * Fix thread safety issue in master service * optimize code.	3 years ago
wangyang	0d8079a816	[DS-7992][feat] Alert module judging strategy (#8636 )	3 years ago
caishunfeng	ddd4474189	change quartz executor to be managed by spring (#8774 ) Co-authored-by: caishunfeng <534328519@qq.com>	3 years ago
Martin Huang	b0fc6e7a69	[Feature-5752][server]Add blocking task (#6272 ) * fix python task execution error * modify code style * delete AbstractTask#setCommand * migrate blocking task alert feature and common attribute * blocking task migrate(complete base features) * migrate blocking task test * resolve TaskNode.java conflict * fix package problem in BlockingParameters * resolve duplicate code * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * resolve code style * optimise code based on pr#6283 in ds master repo * add log * modify log * resolve conflicts * change hardcode to variable * Add blocking task UI * refactor old code to compatable with the newest branch * add override annotation * remove old code * remove never used packages * add missing packages declaration * Refine the i18n description * add UT * resolve some code smell problems * add fetch-depth * enable unit test in sonar analysis * disable unit test * remove unused packages * resolve duplicate codes * add logs when task status changed and rename some variables * add BLOCK & READY_BLOCK status and corresponding processing code * revoke UI * revert unit-test Co-authored-by: Jave-Chen <baicai.chen@gmail.com>	3 years ago
yimaixinchen	a72742b9de	remove unuse import jsonFormat (#8548 )	3 years ago

1 2 3 4 5 ...

379 Commits (1b1a567fec8bc0c5027b2f8ea11bc57a85101ced)