bao liang
5 years ago
committed by
GitHub
146 changed files with 2966 additions and 1449 deletions
@ -0,0 +1,55 @@
|
||||
Easy Scheduler Release 1.1.0 |
||||
=== |
||||
Easy Scheduler 1.1.0是1.x系列中的第六个版本。 |
||||
|
||||
新特性： |
||||
=== |
||||
- [[EasyScheduler-391](https://github.com/analysys/EasyScheduler/issues/391)] run a process under a specified tenement user |
||||
- [[EasyScheduler-288](https://github.com/analysys/EasyScheduler/issues/288)] Feature/qiye_weixin |
||||
- [[EasyScheduler-189](https://github.com/analysys/EasyScheduler/issues/189)] Kerberos等安全支持 |
||||
- [[EasyScheduler-398](https://github.com/analysys/EasyScheduler/issues/398)]管理员，有租户（install.sh设置默认租户），可以创建资源、项目和数据源（限制有一个管理员） |
||||
- [[EasyScheduler-293](https://github.com/analysys/EasyScheduler/issues/293)]点击运行流程时候选择的参数，没有地方可查看，也没有保存 |
||||
- [[EasyScheduler-401](https://github.com/analysys/EasyScheduler/issues/401)]定时很容易定时每秒一次，定时完成以后可以在页面显示一下下次触发时间 |
||||
- [[EasyScheduler-493](https://github.com/analysys/EasyScheduler/pull/493)]add datasource kerberos auth and FAQ modify and add resource upload s3 |
||||
|
||||
|
||||
增强： |
||||
=== |
||||
- [[EasyScheduler-227](https://github.com/analysys/EasyScheduler/issues/227)] upgrade spring-boot to 2.1.x and spring to 5.x |
||||
- [[EasyScheduler-434](https://github.com/analysys/EasyScheduler/issues/434)] worker节点数量 zk和mysql中不一致 |
||||
- [[EasyScheduler-435](https://github.com/analysys/EasyScheduler/issues/435)]邮箱格式的验证 |
||||
- [[EasyScheduler-441](https://github.com/analysys/EasyScheduler/issues/441)] 禁止运行节点加入已完成节点检测 |
||||
- [[EasyScheduler-400](https://github.com/analysys/EasyScheduler/issues/400)] 首页页面，队列统计不和谐，命令统计无数据 |
||||
- [[EasyScheduler-395](https://github.com/analysys/EasyScheduler/issues/395)] 对于容错恢复的流程，状态不能为 **正在运行 |
||||
- [[EasyScheduler-529](https://github.com/analysys/EasyScheduler/issues/529)] optimize poll task from zookeeper |
||||
- [[EasyScheduler-242](https://github.com/analysys/EasyScheduler/issues/242)]worker-server节点获取任务性能问题 |
||||
- [[EasyScheduler-352](https://github.com/analysys/EasyScheduler/issues/352)]worker 分组, 队列消费问题 |
||||
- [[EasyScheduler-461](https://github.com/analysys/EasyScheduler/issues/461)]查看数据源参数，需要加密账号密码信息 |
||||
- [[EasyScheduler-396](https://github.com/analysys/EasyScheduler/issues/396)]Dockerfile优化，并关联Dockerfile和github实现自动打镜像 |
||||
- [[EasyScheduler-389](https://github.com/analysys/EasyScheduler/issues/389)]service monitor cannot find the change of master/worker |
||||
- [[EasyScheduler-511](https://github.com/analysys/EasyScheduler/issues/511)]support recovery process from stop/kill nodes. |
||||
- [[EasyScheduler-399](https://github.com/analysys/EasyScheduler/issues/399)]HadoopUtils指定用户操作，而不是 **部署用户 |
||||
|
||||
修复： |
||||
=== |
||||
- [[EasyScheduler-394](https://github.com/analysys/EasyScheduler/issues/394)] master&worker部署在同一台机器上时，如果重启master&worker服务，会导致之前调度的任务无法继续调度 |
||||
- [[EasyScheduler-469](https://github.com/analysys/EasyScheduler/issues/469)]Fix naming errors,monitor page |
||||
- [[EasyScheduler-392](https://github.com/analysys/EasyScheduler/issues/392)]Feature request: fix email regex check |
||||
- [[EasyScheduler-405](https://github.com/analysys/EasyScheduler/issues/405)]定时修改/添加页面，开始时间和结束时间不能相同 |
||||
- [[EasyScheduler-517](https://github.com/analysys/EasyScheduler/issues/517)]补数 - 子工作流 - 时间参数 |
||||
- [[EasyScheduler-532](https://github.com/analysys/EasyScheduler/issues/532)]python节点不执行的问题 |
||||
- [[EasyScheduler-543](https://github.com/analysys/EasyScheduler/issues/543)]optimize datasource connection params safety |
||||
- [[EasyScheduler-569](https://github.com/analysys/EasyScheduler/issues/569)]定时任务无法真正停止 |
||||
- [[EasyScheduler-463](https://github.com/analysys/EasyScheduler/issues/463)]邮箱验证不支持非常见后缀邮箱 |
||||
|
||||
|
||||
|
||||
|
||||
感谢： |
||||
=== |
||||
最后但最重要的是，没有以下伙伴的贡献就没有新版本的诞生： |
||||
|
||||
Baoqi, jimmy201602, samz406, petersear, millionfor, hyperknob, fanguanqun, yangqinlong, qq389401879, chgxtony, Stanfan, lfyee, thisnew, hujiang75277381, sunnyingit, lgbo-ustc, ivivi, lzy305, JackIllkid, telltime, lipengbo2018, wuchunfu, telltime |
||||
|
||||
以及微信群里众多的热心伙伴！在此非常感谢！ |
||||
|
||||
|
After Width: | Height: | Size: 104 KiB |
|
After Width: | Height: | Size: 36 KiB |
|
After Width: | Height: | Size: 106 KiB |
|
After Width: | Height: | Size: 36 KiB |
@ -0,0 +1,57 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
package cn.escheduler.alert.manager; |
||||
|
||||
import cn.escheduler.alert.utils.Constants; |
||||
import cn.escheduler.alert.utils.EnterpriseWeChatUtils; |
||||
import cn.escheduler.dao.model.Alert; |
||||
import org.slf4j.Logger; |
||||
import org.slf4j.LoggerFactory; |
||||
|
||||
import java.io.IOException; |
||||
import java.util.Arrays; |
||||
import java.util.HashMap; |
||||
import java.util.List; |
||||
import java.util.Map; |
||||
|
||||
/** |
||||
* Enterprise WeChat Manager |
||||
*/ |
||||
public class EnterpriseWeChatManager { |
||||
private static final Logger logger = LoggerFactory.getLogger(MsgManager.class); |
||||
/** |
||||
* Enterprise We Chat send |
||||
* @param alert |
||||
*/ |
||||
public Map<String,Object> send(Alert alert, String token){ |
||||
Map<String,Object> retMap = new HashMap<>(); |
||||
retMap.put(Constants.STATUS, false); |
||||
String agentId = EnterpriseWeChatUtils.enterpriseWeChatAgentId; |
||||
String users = EnterpriseWeChatUtils.enterpriseWeChatUsers; |
||||
List<String> userList = Arrays.asList(users.split(",")); |
||||
logger.info("send message {}",alert); |
||||
String msg = EnterpriseWeChatUtils.makeUserSendMsg(userList, agentId,EnterpriseWeChatUtils.markdownByAlert(alert)); |
||||
try { |
||||
EnterpriseWeChatUtils.sendEnterpriseWeChat(Constants.UTF_8, msg, token); |
||||
} catch (IOException e) { |
||||
logger.error(e.getMessage(),e); |
||||
} |
||||
retMap.put(Constants.STATUS, true); |
||||
return retMap; |
||||
} |
||||
|
||||
} |
||||
@ -1,42 +0,0 @@
|
||||
<!-- Logback configuration. See http://logback.qos.ch/manual/index.html --> |
||||
<configuration scan="true" scanPeriod="120 seconds"> |
||||
<logger name="org.apache.zookeeper" level="WARN"/> |
||||
<logger name="org.apache.hbase" level="WARN"/> |
||||
<logger name="org.apache.hadoop" level="WARN"/> |
||||
|
||||
<property name="log.base" value="logs" /> |
||||
|
||||
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> |
||||
<encoder> |
||||
<pattern> |
||||
[%level] %date{yyyy-MM-dd HH:mm:ss.SSS} %logger{96}:[%line] - %msg%n |
||||
</pattern> |
||||
<charset>UTF-8</charset> |
||||
</encoder> |
||||
</appender> |
||||
|
||||
<appender name="APISERVERLOGFILE" class="ch.qos.logback.core.rolling.RollingFileAppender"> |
||||
<!-- Log level filter --> |
||||
<filter class="ch.qos.logback.classic.filter.ThresholdFilter"> |
||||
<level>INFO</level> |
||||
</filter> |
||||
<file>${log.base}/escheduler-api-server.log</file> |
||||
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy"> |
||||
<fileNamePattern>${log.base}/escheduler-api-server.%d{yyyy-MM-dd_HH}.%i.log</fileNamePattern> |
||||
<maxHistory>168</maxHistory> |
||||
<maxFileSize>64MB</maxFileSize> |
||||
</rollingPolicy> |
||||
|
||||
<encoder> |
||||
<pattern> |
||||
[%level] %date{yyyy-MM-dd HH:mm:ss.SSS} %logger{96}:[%line] - %msg%n |
||||
</pattern> |
||||
<charset>UTF-8</charset> |
||||
</encoder> |
||||
|
||||
</appender> |
||||
|
||||
<root level="INFO"> |
||||
<appender-ref ref="STDOUT" /> |
||||
</root> |
||||
</configuration> |
||||
@ -0,0 +1,29 @@
|
||||
package cn.escheduler.api.utils; |
||||
|
||||
import cn.escheduler.dao.model.MasterServer; |
||||
import org.junit.Assert; |
||||
import org.junit.Test; |
||||
|
||||
|
||||
import java.util.List; |
||||
|
||||
public class ZookeeperMonitorUtilsTest { |
||||
|
||||
|
||||
@Test |
||||
public void testGetMasterLsit(){ |
||||
|
||||
ZookeeperMonitor zookeeperMonitor = new ZookeeperMonitor(); |
||||
|
||||
|
||||
List<MasterServer> masterServerList = zookeeperMonitor.getMasterServers(); |
||||
|
||||
List<MasterServer> workerServerList = zookeeperMonitor.getWorkerServers(); |
||||
|
||||
Assert.assertEquals(masterServerList.size(), 1); |
||||
Assert.assertEquals(workerServerList.size(), 1); |
||||
|
||||
|
||||
} |
||||
|
||||
} |
||||
@ -0,0 +1,29 @@
|
||||
/* |
||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
||||
* contributor license agreements. See the NOTICE file distributed with |
||||
* this work for additional information regarding copyright ownership. |
||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
||||
* (the "License"); you may not use this file except in compliance with |
||||
* the License. You may obtain a copy of the License at |
||||
* |
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
* |
||||
* Unless required by applicable law or agreed to in writing, software |
||||
* distributed under the License is distributed on an "AS IS" BASIS, |
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
||||
* See the License for the specific language governing permissions and |
||||
* limitations under the License. |
||||
*/ |
||||
package cn.escheduler.common.enums; |
||||
|
||||
/** |
||||
* data base types |
||||
*/ |
||||
public enum ResUploadType { |
||||
/** |
||||
* 0 hdfs |
||||
* 1 s3 |
||||
* 2 none |
||||
*/ |
||||
HDFS,S3,NONE |
||||
} |
||||
@ -0,0 +1,15 @@
|
||||
package cn.escheduler.common.enums; |
||||
|
||||
/** |
||||
* zk node type |
||||
*/ |
||||
public enum ZKNodeType { |
||||
|
||||
/** |
||||
* 0 do not send warning; |
||||
* 1 send if process success; |
||||
* 2 send if process failed; |
||||
* 3 send if process ending; |
||||
*/ |
||||
MASTER, WORKER, DEAD_SERVER, TASK_QUEUE; |
||||
} |
||||
@ -0,0 +1,41 @@
|
||||
package cn.escheduler.common.utils; |
||||
|
||||
import org.junit.Assert; |
||||
import org.junit.Test; |
||||
|
||||
import static org.junit.Assert.*; |
||||
|
||||
public class IpUtilsTest { |
||||
|
||||
@Test |
||||
public void ipToLong() { |
||||
|
||||
String ip = "192.168.110.1"; |
||||
String ip2 = "0.0.0.0"; |
||||
long longNumber = IpUtils.ipToLong(ip); |
||||
long longNumber2 = IpUtils.ipToLong(ip2); |
||||
System.out.println(longNumber); |
||||
Assert.assertEquals(longNumber, 3232263681L); |
||||
Assert.assertEquals(longNumber2, 0L); |
||||
|
||||
String ip3 = "255.255.255.255"; |
||||
long longNumber3 = IpUtils.ipToLong(ip3); |
||||
System.out.println(longNumber3); |
||||
Assert.assertEquals(longNumber3, 4294967295L); |
||||
|
||||
} |
||||
|
||||
@Test |
||||
public void longToIp() { |
||||
|
||||
String ip = "192.168.110.1"; |
||||
String ip2 = "0.0.0.0"; |
||||
long longNum = 3232263681L; |
||||
String i1 = IpUtils.longToIp(longNum); |
||||
|
||||
String i2 = IpUtils.longToIp(0); |
||||
|
||||
Assert.assertEquals(ip, i1); |
||||
Assert.assertEquals(ip2, i2); |
||||
} |
||||
} |
||||
@ -0,0 +1,114 @@
|
||||
<template> |
||||
<div class="starting-params-dag-index"> |
||||
<template v-if="isView && isActive"> |
||||
<div class="box"> |
||||
<p class="box-hd"><i class="fa fa-chevron-circle-right"></i><b>{{$t('Startup parameter')}}</b></p> |
||||
<ul class="box-bd"> |
||||
<li><span>{{$t('Startup type')}}：</span><span>{{_rtRunningType(startupParam.commandType)}}</span></li> |
||||
<li><span>{{$t('Complement range')}}：</span><span v-if="startupParam.commandParam && startupParam.commandParam.complementStartDate">{{startupParam.commandParam.complementStartDate}}-{{startupParam.commandParam.complementEndDate}}</span><span v-else>-</span></li> |
||||
<li><span>{{$t('Failure Strategy')}}：</span><span>{{startupParam.failureStrategy === 'END' ? $t('End') : $t('Continue')}}</span></li> |
||||
<li><span>{{$t('Process priority')}}：</span><span>{{startupParam.processInstancePriority}}</span></li> |
||||
<li><span>{{$t('Worker group')}}：</span><span v-if="workerGroupList.length">{{_rtWorkerGroupName(startupParam.workerGroupId)}}</span></li> |
||||
<li><span>{{$t('Notification strategy')}}：</span><span>{{_rtWarningType(startupParam.warningType)}}</span></li> |
||||
<li><span>{{$t('Notification group')}}：</span><span v-if="notifyGroupList.length">{{_rtNotifyGroupName(startupParam.warningGroupId)}}</span></li> |
||||
<li><span>{{$t('Recipient')}}：</span><span>{{startupParam.receivers || '-'}}</span></li> |
||||
<li><span>{{$t('Cc')}}：</span><span>{{startupParam.receiversCc || '-'}}</span></li> |
||||
</ul> |
||||
</div> |
||||
</template> |
||||
</div> |
||||
</template> |
||||
<script> |
||||
import store from '@/conf/home/store' |
||||
import { runningType } from '@/conf/home/pages/dag/_source/config' |
||||
import { warningTypeList } from '@/conf/home/pages/projects/pages/definition/pages/list/_source/util' |
||||
|
||||
export default { |
||||
name: 'starting-params-dag-index', |
||||
data () { |
||||
return { |
||||
store, |
||||
startupParam: store.state.dag.startup, |
||||
isView: false, |
||||
isActive: true, |
||||
notifyGroupList: null, |
||||
workerGroupList: null |
||||
} |
||||
}, |
||||
methods: { |
||||
_toggleParam () { |
||||
this.isView = !this.isView |
||||
}, |
||||
_rtRunningType (code) { |
||||
return _.filter(runningType, v => v.code === code)[0].desc |
||||
}, |
||||
_rtWarningType (id) { |
||||
return _.filter(warningTypeList, v => v.id === id)[0].code |
||||
}, |
||||
_rtNotifyGroupName (id) { |
||||
let o = _.filter(this.notifyGroupList, v => v.id === id) |
||||
if (o && o.length) { |
||||
return o[0].code |
||||
} |
||||
return '-' |
||||
}, |
||||
_rtWorkerGroupName (id) { |
||||
let o = _.filter(this.workerGroupList, v => v.id === id) |
||||
if (o && o.length) { |
||||
return o[0].name |
||||
} |
||||
return '-' |
||||
}, |
||||
_getNotifyGroupList () { |
||||
let notifyGroupListS = _.cloneDeep(this.store.state.dag.notifyGroupListS) || [] |
||||
if (!notifyGroupListS.length) { |
||||
this.store.dispatch('dag/getNotifyGroupList').then(res => { |
||||
this.notifyGroupList = res |
||||
}) |
||||
} else { |
||||
this.notifyGroupList = notifyGroupListS |
||||
} |
||||
}, |
||||
_getWorkerGroupList () { |
||||
let stateWorkerGroupsList = this.store.state.security.workerGroupsListAll || [] |
||||
if (!stateWorkerGroupsList.length) { |
||||
this.store.dispatch('security/getWorkerGroupsAll').then(res => { |
||||
this.workerGroupList = res |
||||
}) |
||||
} else { |
||||
this.workerGroupList = stateWorkerGroupsList |
||||
} |
||||
} |
||||
}, |
||||
watch: { |
||||
'$route': { |
||||
deep: true, |
||||
handler () { |
||||
this.isActive = false |
||||
this.$nextTick(() => (this.isActive = true)) |
||||
} |
||||
} |
||||
}, |
||||
mounted () { |
||||
this._getNotifyGroupList() |
||||
this._getWorkerGroupList() |
||||
} |
||||
} |
||||
</script> |
||||
<style lang="scss"> |
||||
.starting-params-dag-index { |
||||
.box { |
||||
padding: 5px 10px 10px; |
||||
.box-hd { |
||||
.fa { |
||||
color: #0097e0; |
||||
margin-right: 4px; |
||||
} |
||||
font-size: 16px; |
||||
} |
||||
.box-bd { |
||||
margin-left: 20px; |
||||
} |
||||
} |
||||
} |
||||
</style> |
||||
@ -0,0 +1,56 @@
|
||||
<template> |
||||
<x-select |
||||
:disabled="isDetails" |
||||
@on-change="_onChange" |
||||
v-model="value" |
||||
style="width: 180px"> |
||||
<x-option |
||||
v-for="item in itemList" |
||||
:key="item.id" |
||||
:value="item.id" |
||||
:label="item.tenantName"> |
||||
</x-option> |
||||
</x-select> |
||||
</template> |
||||
<script> |
||||
import disabledState from '@/module/mixin/disabledState' |
||||
export default { |
||||
name: 'form-tenant', |
||||
data () { |
||||
return { |
||||
itemList: [] |
||||
} |
||||
}, |
||||
mixins: [disabledState], |
||||
props: { |
||||
value: { |
||||
type: Number, |
||||
default: -1 |
||||
} |
||||
}, |
||||
model: { |
||||
prop: 'value', |
||||
event: 'tenantSelectEvent' |
||||
}, |
||||
methods: { |
||||
_onChange (o) { |
||||
this.value = o.value |
||||
this.$emit('tenantSelectEvent', o.value) |
||||
} |
||||
}, |
||||
watch: { |
||||
}, |
||||
created () { |
||||
let stateTenantAllList = this.store.state.security.tenantAllList || [] |
||||
if (stateTenantAllList.length) { |
||||
this.itemList = stateTenantAllList |
||||
} else { |
||||
this.store.dispatch('security/getTenantList').then(res => { |
||||
this.$nextTick(() => { |
||||
this.itemList = res |
||||
}) |
||||
}) |
||||
} |
||||
} |
||||
} |
||||
</script> |
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue