DolphinScheduler

Commit Graph

Author	SHA1	Message	Date
Eric Gao	f689220290	[Feature][Task Plugin] Increase zeppelin task stability in production (#11010 ) * [Feature][Task Plugin] Increase zeppelin task stability in production (#10584) * Add front-end and update docs for the production mode of zeppelin task * Fix minor front-end bug of zeppelin task plugin * Refactor ZeppelinParameters with lombok * Fix formatting * Replace @Data with @Getter, @Setter, @ToString to avoid decrease in test coverage	2 years ago
xuhhui	35f32cd16e	[Improvement][Task]Improve seatunnel task command log (#11027 )	2 years ago
Eric Gao	2f0d907bcb	[Improvement][Task Plugin]Refactor JupyterParameters.java with lombok (#11045 ) * [Improvement][Task Plugin]Refactor JupyterParameters.java with lombok (#11040)	2 years ago
Wenjun Ruan	5e9c7dad23	Add dolphinscheduler-bom to manage the dependency version (#11025 )	2 years ago
JieguangZhou	11a354927d	[fix] [MLflow Task plugin] Sometimes errors are reported using preset algorithms (#10986 ) (#10987 ) * Optimize the download method of preset project * fix checkstyle	2 years ago
zhuxt2015	7196b125d9	[Feature][Datax Task]Datax task plugin support custom parameters (#10950 ) * datax task plugin support custom parameters	2 years ago
Eric Gao	a38fa34d43	[Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#10925 ) * [Feature][Task Plugin] Enable users to switch endpoints in zeppelin tasks (#9814)	2 years ago
chuxing	427c58546c	[fix][plugin]: fix spark sql use custom parma. (#10821 ) * [fix][plugin]: fix spark sql use custom parma. * [fix][plugin] roll back part of the code Co-authored-by: huaangcg <huangcg@getui.com>	2 years ago
JieguangZhou	02b2de2a1e	[fix] mlflow project success status (#10897 ) close: #10902	2 years ago
Tq	499e5b1307	[bug][task-plugins]fix switch class name and add param check to the switch task plugin (#10894 ) * fix switch class name and add param check to the switch task plugin	2 years ago
ZhaoGuodong	8eaf5a2309	[Feature-10219][EMR] EMR supports use <add-Steps> to add steps to an existing cluster (#10657 ) * Add the ProgramType parameter to distinguish task types * EmrAddStepsTask supports Add-Steps * UI supports Add-Steps * EmrTask modify the name of the class to EmrJobFlowTask * add ERM Task abstract base class AbstractEmrTask * add testcase for EmrAddStepsTask * Modifying help Documents	3 years ago
Eric Gao	71f0168510	[Feature] Enable users to create python env from requirements.txt (#10658 )	3 years ago
Kerwin	bd1be6cf36	[Task] Added Apache SeaTunnel 2.x task node (#10480 )	3 years ago
JieguangZhou	7d79a2165e	[Optimization]Optimize some details of MLFlow task plugin #10740 (#10739 ) * Optimize some details of MLFlow task plugin * Update dolphinscheduler-task-plugin/dolphinscheduler-task-mlflow/src/main/java/org/apache/dolphinscheduler/plugin/task/mlflow/MlflowParameters.java * fix some nips Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>	3 years ago
aiwenmo	5b0347e8da	[Task] Add Dinky task to better support the development and execution of FlinkSQL (#10640 )	3 years ago
Paul Zhang	4a3c3e7797	[feature][task-flink] Support Flink application mode (#9577 )	3 years ago
WangJPLeo	8f621ff98b	[Optimization] Calculate global parameter and local parameter at master. (#10704 ) * Global parameter and local parameter calculation external expansion. * k8s task ut fix. * TimePlaceholderUtils import DateUtils fix * follow the review comments to fix. * follow the review comments to fix. * e2e rerun	3 years ago
zhuxt2015	7e7df1d8ff	[Bug] [Task] SQL task no exception stack trace in task log (#10656 )	3 years ago
Wenjun Ruan	66624c5c86	[Bug] [Master] Worker failover will cause task cannot be failover (#10631 ) * fix worker failover may lose event	3 years ago
Wenjun Ruan	db595b3eff	Optimize master log, use MDC to inject workflow instance id and task instance id in log (#10516 ) * Optimize master log, add workflow instance id and task instance id in log * Use MDC to set the workflow info in log4j * Add workflowInstanceId and taskInstanceId in MDC	3 years ago
JieguangZhou	36e20cdfc8	[Feature][Task Plugin] Add DVC task plugin for MLops scenario (#10372 ) (#10407 )	3 years ago
Eric Gao	64cee03fed	[Feature] Enable users to switch and install conda env in jupyter task (#10337 )	3 years ago
Eric Gao	4be3b877c3	[feature][task] Enable zeppelin schedule a whole zeppelin note (#10434 )	3 years ago
guodong	2c227ab384	[Feature-10363][server] S3 Resource center supports bucket customization (#10433 ) * [Feature-10363][server] S3 Resource center supports bucket customization * Modify the configuration. Sort out HDFS and S3 configurations to distinguish them * Modify the configuration comment to clarify the configuration meaning * Modify official documents to add missing configuration descriptions and clarify configuration meanings and usage * Modify the configuration file in e2E to the correct definition, dolphinscheduler-e2e/dolphinscheduler-e2e-case/src/test/resources/docker/file-manage/common.properties * Modified code to support bucket customization * Check that the bucket on personal permission, Otherwise, the startup fails * S3 increase support resource. Storage. Upload. Base. The base path path * Change S3Utils implementation style to be consistent with HadoopUtils * [Feature-10363][server] S3 Resource center supports bucket customization * Modify code review comments	3 years ago
Wenjun Ruan	ad2646ff1f	Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe (#10479 ) * Fix TaskProcessorFactory#getTaskProcessor get common processor is not thread safe	3 years ago
JieguangZhou	3258438f6e	[Feature] [MLOps] support mlflow deploy with docker compose (#10217 ) * [Feature] [MLOps] support mlflow deploy with docker compose fix doc Update docs/docs/en/guide/task/mlflow.md fix doc Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com> revert cancel modification fix ENV name and docker compose command * fix doc image link * fix testModelsDeployDockerCompose * add docker compose container health check and fix mlflow bug * update docker compose healthcheck timeout	3 years ago
xiangzihao	bbba20d39a	change to common lang3 and fix checkstyle (#10380 )	3 years ago
xiangzihao	b1fb17119b	[Feature-6758][Task] Add limit resource usage for tasks base on cgroup (#10373 )	3 years ago
He Zhao	da3c25dc67	[improve] Using create or replace function in sql task (#10170 )	3 years ago
Wenjun Ruan	0bf4043d5f	Move datasource params to datasouce plugin module (#10353 ) * Move datasource params to datasouce plugin module	3 years ago
kezhenxu94	d80cf21456	Clean up unused dependencies and packaging issues (#9944 )	3 years ago
Eric Gao	a8a5da367a	[Feature][Task Plugin] Add support for dynamic form for zeppelin task plugin (#9977 ) (#10269 )	3 years ago
HuangWei	c5961aed25	[feat][task] Add OpenMLDB task plugin (#10198 )	3 years ago
JieguangZhou	54ab2651bd	[Feature][MLops] Add MLflow Projects (custom project) component (#10175 ) * Add MLflow Projects (custom project) component * fix something * update document * update document * Supplementary test cases	3 years ago
JieguangZhou	e11373b963	[Feature][MLops] Support MLflow Models to deploy model service (MLflow models serve and Docker) (#10150 ) * Add backend of MLflow Moldes component * Add front end of MLflow Moldes component * [DOC] update mlflow document * revert unnecessary * fix doc image url * Update docs/docs/en/guide/task/mlflow.md Discard the lr abbreviation Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com> Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>	3 years ago
JieguangZhou	9ab3bebc90	[Feature][Task Plugin] Add mlflow task plugin for MLops scenario. (#10036 ) * [Feature][Task Plugin] Add mlflow task plugin (#9725) * [Feature][Task Plugin] Add mlflow task plugin UI (#9725) * [Feature][Task Plugin] fix license header (#9725) * [Feature][Task Plugin] fix license header (#9725) * [Feature][Task Plugin] revert unnecessary * [Feature][Task Plugin] add auto ml to mlflow task plugin * [DOC] add mlflow document * [DOC] fix mlflow docs imgs * [DOC] fix dead link localhost:5000 * [DOC] fix dead link localhost:5000 * [DOC] remove dead link localhost:5000 * Update docs/docs/en/guide/task/mlflow.md * Update docs/docs/zh/guide/task/mlflow.md * [DOC] format ui code * [DOC] remove dead link localhost:5000 * [Feature][Task Plugin] revert unnecessary * fix some nits * Move the personal repository to the public repository. * Run the command directly instead of saving the file * fix paramsMap initialize * revert unnecessary Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>	3 years ago
He Zhao	5bb1eb04fc	[Feature][plugin] Add k8s task in task plugin (#9425 ) * [Feature][plugin] Add k8s task in task plugin * [Feature][plugin] fix dos and code problems * [Feature][plugin] refactor some code based on sonar * [Feature][UI] front-end for k8s task plugin * [Feature][plugin] delete some front files * [Feature][plugin] update document * Update docs/docs/zh/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * Update docs/docs/en/guide/task/k8s.md * [Feature][UI] front-end change from review comments * [Feature][UI] replace get namespace list api * [Feature][plugin] change file name * Add kubernetes to zh task list Co-authored-by: hezhao2 <hezhao2@cisco.com> Co-authored-by: William Tong <weitong@cisco.com> Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com>	3 years ago
Dannila	d643e1c1cf	[Fix-10039] Flink run command when perfecting Python jobs (#10042 ) * [fix] flink task * [fix] flink task	3 years ago
zixi0825	359cbe2a1e	[dataquality] Fix task commnd null bug (#9974 )	3 years ago
chuxing	df04c4aef0	[fix-9991][worker]fix statement is closed before resultSet.getMetaData() (#10014 )	3 years ago
Eric Gao	80369363c9	[feat][task plugin] Add Jupyter task plugin (#9872 ) Co-authored-by: Amy0104 <97265214+Amy0104@users.noreply.github.com>	3 years ago
Dannila	2d36449444	[Feature- 9837][plugin/ui] support FlinkSQL Task (#9840 ) * flink_sql * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * '[refactor]flinksql' * [refactor] flinksql * '[refactor]flinksql' * [refactor] dolphinscheduler flinksql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * [refactor] flink sql * Update docs/docs/en/guide/task/flink.md * Update docs/docs/zh/guide/task/flink.md * prettier front-end code Co-authored-by: Jiajie Zhong <zhongjiajie955@gmail.com> Co-authored-by: devosend <devosend@gmail.com>	3 years ago
Kerwin	0e7c98d465	[Feature-9177][Task] The sql task supports configuring segmentation notation to provide execution of multiple statements (#9917 ) * Support sql segmentation to execute multiple functions.	3 years ago
sq-q	ed425d2d2d	[Feature-9772][plugin/ui] support SparkSQL Task (#9790 ) * [refactor] Dolphinscheduler sparkSQL * [refactor] Dolphinscheduler SparkSQL * [refactor] Dolphinscheduler SparkSQL * [refactor] Dolphinscheduler SparkSQL * [refactor] Dolphinscheduler SparkSQL * [refactor] Dolphinscheduler SparkSQL * [refactor] Dolphinscheduler SparkSQL * [refactor] Refactor ui code and add sparksql test cases * [refactor] refactor dolphinscheduler SparkSQL * [refactor] refactor dolphinscheduler plugin-sparkSQL * [refactor] refactor dolphinscheduler plugin-SparkSQL * [refactor] dolphinscheduler plugin-SparkTaskTest * [refactor] dolphinscheduler plugin-SparkTask * [refactor] dolphinscheduler plugin-Spark * [refactor] dolphinscheduler plugin-SparkTask-SparkSQL * [refactor] dolphinscheduler plugin-spark-SparkTask * [refactor] dolphinscheduler plugin-spark-SparkTask redefine code	3 years ago
zhuxt2015	206b7c1c51	[Fix-9352][datasource-api]data source client uses a cache with an expiration time (#9353 ) * datasource client cache with expiration time * remove unused import * fixed unit test * fix * fix * fix * fix * fix * fix * fix	3 years ago
exmy	36f01155b5	[Improvement][server] varPool support syntax #{setValue(key=value)} (#9586 )	3 years ago
Tq	c5b7e5adff	[Bug] [API-9558]fix homepage task instance count method to use submit time to recount (#9559 ) * fix homepage task instance count method to use submit time to recount * fix homepage task instance count method to use submit time to recount * fix homepage task instance count method to use submit time to recount * fix homepage task instance count method JUNIT * fix homepage task instance count method JUNIT * fix homepage task instance count method JUNIT	3 years ago
Eric Gao	a5bbf7852d	[Feature][Task-Plugin]Add zeppelin task-plugin to support Apache Zeppelin (#9327 )	3 years ago
xiangzihao	7f41a96fc1	[Fix-9525] [Worker] Environment did not work as expected (#9527 ) * fix #9525 * change to ${PYTHON_HOME} * remove import * fix ut error	3 years ago
Tq	04d4e4e0c4	fix the OUT direct param could pass to the current script (#9463 )	3 years ago

... 2 3 4 5 6

276 Commits (428f1559a35ee327acb71e8da04e619b480094ea)