You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
1 line
588 KiB
1 line
588 KiB
6 years ago
|
{"index":{"version":"0.5.12","fields":[{"name":"title","boost":10},{"name":"keywords","boost":15},{"name":"body","boost":1}],"ref":"url","documentStore":{"store":{"./":["+","/","200px;","510570367",":","add","airflow","azkaban","balanc","bashoperator、dummyoperator、mysqloperator、hiveoperator、emailoperator、httpoperator、sqloper","big","celeri","cpu","dag监控界面","dask","data","db","develop","easi","easyschedul","easyscheduler上的用户可以通过租户和hadoop用户实现多对一或一对一的映射关系，这对大数据作业的调度是非常重要的。","easyscheduler简介","executor水平扩展","fastest","ha额外要求","issues,","load","load，memory，cpu在线查看","meso","mr、spark、sql(mysql、postgresql、hive、sparksql)、python、procedure、sub_process","respons","schedul","shell、gobblin、hadoopjava、java、hive、pig、spark、hdfstoteradata、teradatatohdf","submit","tabl","th:first","type","way","wechat","width:","xxx","{","}","一个分布式易扩展的可视化dag工作流任务调度系统。致力于解决数据处理流程中错综复杂的依赖关系，使调度系统在数据处理流程中开箱即用。","一键部署","不能直观区分任务类型","不需要(本身就支持ha)","与同类调度系统的对比","以dag图的方式将task按照任务的依赖关系关联起来，可实时可视化监控任务的运行状态","任务太多时会卡死服务器","任务状态、任务类型、重试次数、任务运行机器、可视化变量等关键信息一目了然","任务类型","任务队列机制，单个机器上可调度的任务数量可以灵活配置，当任务过多时会缓存在任务队列中，不会造成机器卡死","使用手册","其主要目标如下：","前端部署文档","功能","单一调度程序","单个web和调度程序组合","单点故障","去中心化的多master和多work","只能看到任务状态","可视化流程定义","后端部署文档","否","契合度","实现集群ha，通过zookeeper实现master集群和worker集群去中心化","帮助","快速部署","所有流程定义操作都是可视化的，通过拖拽任务来绘制dag,配置数据源及资源。同时对于第三方系统，提供api方式的操作。","扩展性","支持","支持丰富的任务类型：shell、mr、spark、sql(mysql、postgresql、hive、sparksql),python,sub_process、procedure等","支持任务日志在线查看及滚动、在线下载日志等","支持传统的shell任务，同时支持大数据平台任务调度：","支持国际化","支持多租户","支持大数据作业spark,hive,mr的调度，同时由于支持多租户，与大数据业务更加契合","支持对master/work","支持工作流优先级、任务优先级及任务的故障转移及任务超时告警/失败","支持工作流全局参数及节点自定义参数设置","支持工作流定时调度、依赖调度、手动调度、手动暂停/停止/恢复，同时支持失败重试/告警、从指定节点恢复失败、kill任务等操作","支持工作流运行历史树形/甘特图展示、支持任务状态统计、流程状态统计","支持暂停，恢复操作","支持补数","支持资源文件的在线上传/下载，管理等，支持在线文件创建、编辑","文档","易用性","是","是否支持多租户","是否支持自定义任务类型","是否支持集群扩展","是否能暂停和恢复","是，但是复杂","更多文档请参考","由于不支持多租户，在大数据平台业务使用不够灵活","稳定性","系统部分截图","设计特点：","调度器使用分布式调度，整体的调度能力会随便集群的规模线性增长，master和worker支持动态上下线","过载处理","还有更多等待伙伴们探索","通过python代码来绘制dag，使用不便，特别是对不会写代码的业务人员基本无法使用。","通过自定义dsl绘制dag并打包上传","部署文档","集群化部署复杂","需将工作流杀死再运行"],"frontend-deploy.html":["\"","\"#\"","\"$1\"","\"upgrade\";","\"usage:","#","#!/b
|