xiangzihao
4 months ago
committed by
GitHub
130 changed files with 282 additions and 5966 deletions
@ -1,5 +1,5 @@ |
|||||||
# Resource Center Introduction |
# Resource Center Introduction |
||||||
|
|
||||||
The Resource Center is typically used for uploading files, UDF functions, and task group management. For a stand-alone |
The Resource Center is typically used for uploading files and task group management. For a stand-alone |
||||||
environment, you can select the local file directory as the upload folder (**this operation does not require Hadoop or HDFS deployment**). |
environment, you can select the local file directory as the upload folder (**this operation does not require Hadoop or HDFS deployment**). |
||||||
Of course, you can also choose to upload to Hadoop or MinIO cluster. In this case, you need to have Hadoop (2.6+) or MinIO and other related environments. |
Of course, you can also choose to upload to Hadoop or MinIO cluster. In this case, you need to have Hadoop (2.6+) or MinIO and other related environments. |
||||||
|
|||||||
@ -1,45 +0,0 @@ |
|||||||
# UDF Manage |
|
||||||
|
|
||||||
## Resource Management |
|
||||||
|
|
||||||
- The resource management and file management functions are similar. The difference is that the resource management is the UDF upload function, and the file management uploads the user programs, scripts and configuration files. |
|
||||||
- It mainly includes the following operations: rename, download, delete, etc. |
|
||||||
- Upload UDF resources: Same as uploading files. |
|
||||||
|
|
||||||
## Function Management |
|
||||||
|
|
||||||
### Create UDF function |
|
||||||
|
|
||||||
Click `Create UDF Function`, enter the UDF function parameters, select the UDF resource, and click `Submit` to create the UDF function. |
|
||||||
Currently only temporary UDF functions for HIVE are supported. |
|
||||||
|
|
||||||
- UDF function name: Enter the name of the UDF function. |
|
||||||
- Package name Class name: Enter the full path of the UDF function. |
|
||||||
- UDF resource: Set the resource file corresponding to the created UDF function. |
|
||||||
|
|
||||||
 |
|
||||||
|
|
||||||
## Example |
|
||||||
|
|
||||||
### Write UDF functions |
|
||||||
|
|
||||||
Users can customize the desired UDF function according to actual production requirements. Here's a function that appends "HelloWorld" to the end of any string. As shown below: |
|
||||||
|
|
||||||
 |
|
||||||
|
|
||||||
### Configure the UDF function |
|
||||||
|
|
||||||
Before configuring UDF functions, you need to upload the required function jar package through resource management. Then enter the function management and configure the relevant information. As shown below: |
|
||||||
|
|
||||||
 |
|
||||||
|
|
||||||
### Use UDF functions |
|
||||||
|
|
||||||
In the process of using UDF functions, users only need to pay attention to the specific function writing, and upload the configuration through the resource center. The system will automatically configure the create function statement, refer to the following: [SqlTask](https://github.com/apache/dolphinscheduler/blob/923f3f38e3271d7f1d22b3abc3497cecb6957e4a/dolphinscheduler-task-plugin/dolphinscheduler-task-sql/src/main/java/org/apache/dolphinscheduler/plugin/task/sql/SqlTask.java#L507-L531) |
|
||||||
|
|
||||||
Enter the workflow to define an SQL node, the data source type is HIVE, and the data source instance type is HIVE/IMPALA. |
|
||||||
|
|
||||||
- SQL statement: `select HwUdf("abc");` This function is used in the same way as the built-in functions, and can be accessed directly using the function name. |
|
||||||
- UDF function: Select the one configured for the resource center. |
|
||||||
|
|
||||||
 |
|
||||||
@ -1,4 +1,4 @@ |
|||||||
# 资源中心简介 |
# 资源中心简介 |
||||||
|
|
||||||
资源中心通常用于上传文件、UDF 函数和任务组管理。 对于 standalone 环境，可以选择本地文件目录作为上传文件夹（此操作不需要Hadoop部署）。当然，你也可以 |
资源中心通常用于上传文件和任务组管理。 对于 standalone 环境，可以选择本地文件目录作为上传文件夹（此操作不需要Hadoop部署）。当然，你也可以 |
||||||
选择上传到 Hadoop 或者 MinIO 集群。 在这种情况下，您需要有 Hadoop（2.6+）或 MinIO 等相关环境。 |
选择上传到 Hadoop 或者 MinIO 集群。 在这种情况下，您需要有 Hadoop（2.6+）或 MinIO 等相关环境。 |
||||||
|
|||||||
@ -1,46 +0,0 @@ |
|||||||
# UDF 管理 |
|
||||||
|
|
||||||
- 资源管理和文件管理功能类似，不同之处是资源管理是上传的 UDF 函数，文件管理上传的是用户程序，脚本及配置文件。 |
|
||||||
- 主要包括以下操作：重命名、下载、删除等。 |
|
||||||
* 上传 UDF 资源 |
|
||||||
|
|
||||||
> 和上传文件相同。 |
|
||||||
|
|
||||||
## 函数管理 |
|
||||||
|
|
||||||
* 创建 UDF 函数 |
|
||||||
|
|
||||||
> 点击“创建 UDF 函数”，输入 UDF 函数参数，选择udf资源，点击“提交”，创建 UDF 函数。 |
|
||||||
> 目前只支持 HIVE 的临时 UDF 函数 |
|
||||||
|
|
||||||
- UDF 函数名称：输入 UDF 函数时的名称 |
|
||||||
- 包名类名：输入 UDF 函数的全路径 |
|
||||||
- UDF 资源：设置创建的 UDF 对应的资源文件 |
|
||||||
|
|
||||||
 |
|
||||||
|
|
||||||
## 任务样例 |
|
||||||
|
|
||||||
### 编写 UDF 函数 |
|
||||||
|
|
||||||
用户可以根据实际生产需求，自定义想要的 UDF 函数。这里编写一个在任意字符串的末尾添加 "HelloWorld" 的函数。如下图所示： |
|
||||||
|
|
||||||
 |
|
||||||
|
|
||||||
### 配置 UDF 函数 |
|
||||||
|
|
||||||
配置 UDF 函数前，需要先通过资源管理上传所需的函数 jar 包。然后进入函数管理，配置相关信息即可。如下图所示： |
|
||||||
|
|
||||||
 |
|
||||||
|
|
||||||
### 使用 UDF 函数 |
|
||||||
|
|
||||||
在使用 UDF 函数过程中，用户只需关注具体的函数编写，通过资源中心上传配置完成即可。系统会自动配置 create function 语句，参考如下：[SqlTask](https://github.com/apache/dolphinscheduler/blob/923f3f38e3271d7f1d22b3abc3497cecb6957e4a/dolphinscheduler-task-plugin/dolphinscheduler-task-sql/src/main/java/org/apache/dolphinscheduler/plugin/task/sql/SqlTask.java#L507-L531) |
|
||||||
|
|
||||||
进入工作流定义一个 SQL 节点，数据源类型选择为 HIVE，数据源实例类型为 HIVE/IMPALA。 |
|
||||||
|
|
||||||
- SQL 语句：`select HwUdf("abc");` 该函数与内置函数使用方式一样，直接使用函数名称即可访问。 |
|
||||||
- UDF 函数：选择资源中心所配置的即可。 |
|
||||||
|
|
||||||
 |
|
||||||
|
|
||||||
|
Before Width: | Height: | Size: 79 KiB |
|
Before Width: | Height: | Size: 16 KiB |
|
Before Width: | Height: | Size: 82 KiB |
|
Before Width: | Height: | Size: 132 KiB |
@ -1,46 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.api.audit.operator.impl; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.api.audit.operator.BaseAuditOperator; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc; |
|
||||||
import org.apache.dolphinscheduler.dao.mapper.UdfFuncMapper; |
|
||||||
|
|
||||||
import org.apache.commons.lang3.math.NumberUtils; |
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired; |
|
||||||
import org.springframework.stereotype.Service; |
|
||||||
|
|
||||||
@Service |
|
||||||
public class UdfFunctionAuditOperatorImpl extends BaseAuditOperator { |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UdfFuncMapper udfFuncMapper; |
|
||||||
|
|
||||||
@Override |
|
||||||
protected String getObjectNameFromIdentity(Object identity) { |
|
||||||
int objId = NumberUtils.toInt(identity.toString(), -1); |
|
||||||
if (objId == -1) { |
|
||||||
return ""; |
|
||||||
} |
|
||||||
|
|
||||||
UdfFunc obj = udfFuncMapper.selectUdfById(objId); |
|
||||||
return obj == null ? "" : obj.getFuncName(); |
|
||||||
} |
|
||||||
|
|
||||||
} |
|
||||||
@ -1,118 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.api.service; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.api.utils.Result; |
|
||||||
import org.apache.dolphinscheduler.common.enums.UdfType; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.User; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf func service |
|
||||||
*/ |
|
||||||
public interface UdfFuncService { |
|
||||||
|
|
||||||
/** |
|
||||||
* create udf function |
|
||||||
* |
|
||||||
* @param loginUser login user |
|
||||||
* @param type udf type |
|
||||||
* @param funcName function name |
|
||||||
* @param argTypes argument types |
|
||||||
* @param database database |
|
||||||
* @param desc description |
|
||||||
* @param className class name |
|
||||||
* @return create result code |
|
||||||
*/ |
|
||||||
Result<Object> createUdfFunction(User loginUser, |
|
||||||
String funcName, |
|
||||||
String className, |
|
||||||
String fullName, |
|
||||||
String argTypes, |
|
||||||
String database, |
|
||||||
String desc, |
|
||||||
UdfType type); |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf function |
|
||||||
* |
|
||||||
* @param id udf function id |
|
||||||
* @return udf function detail |
|
||||||
*/ |
|
||||||
Result<Object> queryUdfFuncDetail(User loginUser, int id); |
|
||||||
|
|
||||||
/** |
|
||||||
* updateProcessInstance udf function |
|
||||||
* |
|
||||||
* @param udfFuncId udf function id |
|
||||||
* @param type resource type |
|
||||||
* @param funcName function name |
|
||||||
* @param argTypes argument types |
|
||||||
* @param database data base |
|
||||||
* @param desc description |
|
||||||
* @param resourceId resource id |
|
||||||
* @param fullName resource full name |
|
||||||
* @param className class name |
|
||||||
* @return update result code |
|
||||||
*/ |
|
||||||
Result<Object> updateUdfFunc(User loginUser, |
|
||||||
int udfFuncId, |
|
||||||
String funcName, |
|
||||||
String className, |
|
||||||
String argTypes, |
|
||||||
String database, |
|
||||||
String desc, |
|
||||||
UdfType type, |
|
||||||
String fullName); |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf function list paging |
|
||||||
* |
|
||||||
* @param loginUser login user |
|
||||||
* @param pageNo page number |
|
||||||
* @param pageSize page size |
|
||||||
* @param searchVal search value |
|
||||||
* @return udf function list page |
|
||||||
*/ |
|
||||||
Result queryUdfFuncListPaging(User loginUser, String searchVal, Integer pageNo, Integer pageSize); |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf list |
|
||||||
* |
|
||||||
* @param loginUser login user |
|
||||||
* @param type udf type |
|
||||||
* @return udf func list |
|
||||||
*/ |
|
||||||
Result<Object> queryUdfFuncList(User loginUser, Integer type); |
|
||||||
|
|
||||||
/** |
|
||||||
* delete udf function |
|
||||||
* |
|
||||||
* @param id udf function id |
|
||||||
* @return delete result code |
|
||||||
*/ |
|
||||||
Result<Object> delete(User loginUser, int id); |
|
||||||
|
|
||||||
/** |
|
||||||
* verify udf function by name |
|
||||||
* |
|
||||||
* @param name name |
|
||||||
* @return true if the name can user, otherwise return false |
|
||||||
*/ |
|
||||||
Result<Object> verifyUdfFuncByName(User loginUser, String name); |
|
||||||
|
|
||||||
} |
|
||||||
@ -1,399 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.api.service.impl; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant; |
|
||||||
import org.apache.dolphinscheduler.api.enums.Status; |
|
||||||
import org.apache.dolphinscheduler.api.service.UdfFuncService; |
|
||||||
import org.apache.dolphinscheduler.api.utils.PageInfo; |
|
||||||
import org.apache.dolphinscheduler.api.utils.Result; |
|
||||||
import org.apache.dolphinscheduler.common.enums.AuthorizationType; |
|
||||||
import org.apache.dolphinscheduler.common.enums.UdfType; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.User; |
|
||||||
import org.apache.dolphinscheduler.dao.mapper.UDFUserMapper; |
|
||||||
import org.apache.dolphinscheduler.dao.mapper.UdfFuncMapper; |
|
||||||
import org.apache.dolphinscheduler.plugin.storage.api.StorageOperator; |
|
||||||
|
|
||||||
import org.apache.commons.lang3.StringUtils; |
|
||||||
|
|
||||||
import java.util.ArrayList; |
|
||||||
import java.util.Collections; |
|
||||||
import java.util.Date; |
|
||||||
import java.util.List; |
|
||||||
import java.util.Set; |
|
||||||
|
|
||||||
import lombok.extern.slf4j.Slf4j; |
|
||||||
|
|
||||||
import org.springframework.beans.factory.annotation.Autowired; |
|
||||||
import org.springframework.stereotype.Service; |
|
||||||
import org.springframework.transaction.annotation.Transactional; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.core.metadata.IPage; |
|
||||||
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf func service impl |
|
||||||
*/ |
|
||||||
@Service |
|
||||||
@Slf4j |
|
||||||
public class UdfFuncServiceImpl extends BaseServiceImpl implements UdfFuncService { |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UdfFuncMapper udfFuncMapper; |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UDFUserMapper udfUserMapper; |
|
||||||
|
|
||||||
@Autowired(required = false) |
|
||||||
private StorageOperator storageOperator; |
|
||||||
|
|
||||||
/** |
|
||||||
* create udf function |
|
||||||
* |
|
||||||
* @param loginUser login user |
|
||||||
* @param type udf type |
|
||||||
* @param funcName function name |
|
||||||
* @param argTypes argument types |
|
||||||
* @param database database |
|
||||||
* @param desc description |
|
||||||
* @param className class name |
|
||||||
* @return create result code |
|
||||||
*/ |
|
||||||
@Override |
|
||||||
@Transactional |
|
||||||
public Result<Object> createUdfFunction(User loginUser, |
|
||||||
String funcName, |
|
||||||
String className, |
|
||||||
String fullName, |
|
||||||
String argTypes, |
|
||||||
String database, |
|
||||||
String desc, |
|
||||||
UdfType type) { |
|
||||||
Result<Object> result = new Result<>(); |
|
||||||
|
|
||||||
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.UDF, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_CREATE); |
|
||||||
if (!canOperatorPermissions) { |
|
||||||
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION); |
|
||||||
return result; |
|
||||||
} |
|
||||||
if (checkDescriptionLength(desc)) { |
|
||||||
log.warn("Parameter description is too long."); |
|
||||||
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
// verify udf func name exist
|
|
||||||
if (checkUdfFuncNameExists(funcName)) { |
|
||||||
log.warn("Udf function with the same name already exists."); |
|
||||||
putMsg(result, Status.UDF_FUNCTION_EXISTS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
boolean existResource = storageOperator.exists(fullName); |
|
||||||
|
|
||||||
if (!existResource) { |
|
||||||
log.error("resource full name {} is not exist", fullName); |
|
||||||
putMsg(result, Status.RESOURCE_NOT_EXIST); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
// save data
|
|
||||||
UdfFunc udf = new UdfFunc(); |
|
||||||
Date now = new Date(); |
|
||||||
udf.setUserId(loginUser.getId()); |
|
||||||
udf.setFuncName(funcName); |
|
||||||
udf.setClassName(className); |
|
||||||
if (!StringUtils.isEmpty(argTypes)) { |
|
||||||
udf.setArgTypes(argTypes); |
|
||||||
} |
|
||||||
if (!StringUtils.isEmpty(database)) { |
|
||||||
udf.setDatabase(database); |
|
||||||
} |
|
||||||
udf.setDescription(desc); |
|
||||||
// set resourceId to -1 because we do not store resource to db anymore, instead we use fullName
|
|
||||||
udf.setResourceId(-1); |
|
||||||
udf.setResourceName(fullName); |
|
||||||
udf.setType(type); |
|
||||||
|
|
||||||
udf.setCreateTime(now); |
|
||||||
udf.setUpdateTime(now); |
|
||||||
|
|
||||||
udfFuncMapper.insert(udf); |
|
||||||
log.info("UDF function create complete, udfFuncName:{}.", udf.getFuncName()); |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* |
|
||||||
* @param name name |
|
||||||
* @return check result code |
|
||||||
*/ |
|
||||||
private boolean checkUdfFuncNameExists(String name) { |
|
||||||
List<UdfFunc> resource = udfFuncMapper.queryUdfByIdStr(null, name); |
|
||||||
return resource != null && !resource.isEmpty(); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf function |
|
||||||
* |
|
||||||
* @param id udf function id |
|
||||||
* @return udf function detail |
|
||||||
*/ |
|
||||||
@Override |
|
||||||
public Result<Object> queryUdfFuncDetail(User loginUser, int id) { |
|
||||||
Result<Object> result = new Result<>(); |
|
||||||
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{id}, AuthorizationType.UDF, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW); |
|
||||||
if (!canOperatorPermissions) { |
|
||||||
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION); |
|
||||||
return result; |
|
||||||
} |
|
||||||
UdfFunc udfFunc = udfFuncMapper.selectById(id); |
|
||||||
if (udfFunc == null) { |
|
||||||
log.error("Resource does not exist, udf func id:{}.", id); |
|
||||||
putMsg(result, Status.RESOURCE_NOT_EXIST); |
|
||||||
return result; |
|
||||||
} |
|
||||||
result.setData(udfFunc); |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* updateProcessInstance udf function |
|
||||||
* |
|
||||||
* @param udfFuncId udf function id |
|
||||||
* @param type resource type |
|
||||||
* @param funcName function name |
|
||||||
* @param argTypes argument types |
|
||||||
* @param database data base |
|
||||||
* @param desc description |
|
||||||
* @param fullName resource full name |
|
||||||
* @param className class name |
|
||||||
* @return update result code |
|
||||||
*/ |
|
||||||
@Override |
|
||||||
public Result<Object> updateUdfFunc(User loginUser, |
|
||||||
int udfFuncId, |
|
||||||
String funcName, |
|
||||||
String className, |
|
||||||
String argTypes, |
|
||||||
String database, |
|
||||||
String desc, |
|
||||||
UdfType type, |
|
||||||
String fullName) { |
|
||||||
Result<Object> result = new Result<>(); |
|
||||||
|
|
||||||
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{udfFuncId}, |
|
||||||
AuthorizationType.UDF, ApiFuncIdentificationConstant.UDF_FUNCTION_UPDATE); |
|
||||||
if (!canOperatorPermissions) { |
|
||||||
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION); |
|
||||||
return result; |
|
||||||
} |
|
||||||
if (checkDescriptionLength(desc)) { |
|
||||||
log.warn("Parameter description is too long."); |
|
||||||
putMsg(result, Status.DESCRIPTION_TOO_LONG_ERROR); |
|
||||||
return result; |
|
||||||
} |
|
||||||
// verify udfFunc is exist
|
|
||||||
UdfFunc udf = udfFuncMapper.selectUdfById(udfFuncId); |
|
||||||
|
|
||||||
if (udf == null) { |
|
||||||
log.error("UDF function does not exist, udfFuncId:{}.", udfFuncId); |
|
||||||
result.setCode(Status.UDF_FUNCTION_NOT_EXIST.getCode()); |
|
||||||
result.setMsg(Status.UDF_FUNCTION_NOT_EXIST.getMsg()); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
// verify udfFuncName is exist
|
|
||||||
if (!funcName.equals(udf.getFuncName())) { |
|
||||||
if (checkUdfFuncNameExists(funcName)) { |
|
||||||
log.warn("Udf function exists, can not create again, udfFuncName:{}.", funcName); |
|
||||||
result.setCode(Status.UDF_FUNCTION_EXISTS.getCode()); |
|
||||||
result.setMsg(Status.UDF_FUNCTION_EXISTS.getMsg()); |
|
||||||
return result; |
|
||||||
} |
|
||||||
} |
|
||||||
|
|
||||||
Boolean doesResExist = false; |
|
||||||
try { |
|
||||||
doesResExist = storageOperator.exists(fullName); |
|
||||||
} catch (Exception e) { |
|
||||||
log.error("udf resource :{} checking error", fullName, e); |
|
||||||
result.setCode(Status.RESOURCE_NOT_EXIST.getCode()); |
|
||||||
result.setMsg(Status.RESOURCE_NOT_EXIST.getMsg()); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
if (!doesResExist) { |
|
||||||
log.error("resource full name {} is not exist", fullName); |
|
||||||
result.setCode(Status.RESOURCE_NOT_EXIST.getCode()); |
|
||||||
result.setMsg(Status.RESOURCE_NOT_EXIST.getMsg()); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
Date now = new Date(); |
|
||||||
udf.setFuncName(funcName); |
|
||||||
udf.setClassName(className); |
|
||||||
udf.setArgTypes(argTypes); |
|
||||||
if (!StringUtils.isEmpty(database)) { |
|
||||||
udf.setDatabase(database); |
|
||||||
} |
|
||||||
udf.setDescription(desc); |
|
||||||
// set resourceId to -1 because we do not store resource to db anymore, instead we use fullName
|
|
||||||
udf.setResourceId(-1); |
|
||||||
udf.setResourceName(fullName); |
|
||||||
udf.setType(type); |
|
||||||
|
|
||||||
udf.setUpdateTime(now); |
|
||||||
|
|
||||||
udfFuncMapper.updateById(udf); |
|
||||||
log.info("UDF function update complete, udfFuncId:{}, udfFuncName:{}.", udfFuncId, funcName); |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf function list paging |
|
||||||
* |
|
||||||
* @param loginUser login user |
|
||||||
* @param pageNo page number |
|
||||||
* @param pageSize page size |
|
||||||
* @param searchVal search value |
|
||||||
* @return udf function list page |
|
||||||
*/ |
|
||||||
@Override |
|
||||||
public Result<Object> queryUdfFuncListPaging(User loginUser, String searchVal, Integer pageNo, Integer pageSize) { |
|
||||||
Result<Object> result = new Result(); |
|
||||||
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.UDF, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW); |
|
||||||
if (!canOperatorPermissions) { |
|
||||||
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION); |
|
||||||
return result; |
|
||||||
} |
|
||||||
PageInfo<UdfFunc> pageInfo = new PageInfo<>(pageNo, pageSize); |
|
||||||
IPage<UdfFunc> udfFuncList = getUdfFuncsPage(loginUser, searchVal, pageSize, pageNo); |
|
||||||
pageInfo.setTotal((int) udfFuncList.getTotal()); |
|
||||||
pageInfo.setTotalList(udfFuncList.getRecords()); |
|
||||||
result.setData(pageInfo); |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* get udf functions |
|
||||||
* |
|
||||||
* @param loginUser login user |
|
||||||
* @param searchVal search value |
|
||||||
* @param pageSize page size |
|
||||||
* @param pageNo page number |
|
||||||
* @return udf function list page |
|
||||||
*/ |
|
||||||
private IPage<UdfFunc> getUdfFuncsPage(User loginUser, String searchVal, Integer pageSize, int pageNo) { |
|
||||||
Set<Integer> udfFuncIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.UDF, |
|
||||||
loginUser.getId(), log); |
|
||||||
Page<UdfFunc> page = new Page<>(pageNo, pageSize); |
|
||||||
if (udfFuncIds.isEmpty()) { |
|
||||||
return page; |
|
||||||
} |
|
||||||
return udfFuncMapper.queryUdfFuncPaging(page, new ArrayList<>(udfFuncIds), searchVal); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf list |
|
||||||
* |
|
||||||
* @param loginUser login user |
|
||||||
* @param type udf type |
|
||||||
* @return udf func list |
|
||||||
*/ |
|
||||||
@Override |
|
||||||
public Result<Object> queryUdfFuncList(User loginUser, Integer type) { |
|
||||||
Result<Object> result = new Result<>(); |
|
||||||
|
|
||||||
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.UDF, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW); |
|
||||||
if (!canOperatorPermissions) { |
|
||||||
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION); |
|
||||||
return result; |
|
||||||
} |
|
||||||
Set<Integer> udfFuncIds = resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.UDF, |
|
||||||
loginUser.getId(), log); |
|
||||||
if (udfFuncIds.isEmpty()) { |
|
||||||
result.setData(Collections.emptyList()); |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
List<UdfFunc> udfFuncList = udfFuncMapper.getUdfFuncByType(new ArrayList<>(udfFuncIds), type); |
|
||||||
|
|
||||||
result.setData(udfFuncList); |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* delete udf function |
|
||||||
* |
|
||||||
* @param id udf function id |
|
||||||
* @return delete result code |
|
||||||
*/ |
|
||||||
@Override |
|
||||||
@Transactional |
|
||||||
public Result<Object> delete(User loginUser, int id) { |
|
||||||
Result<Object> result = new Result<>(); |
|
||||||
|
|
||||||
boolean canOperatorPermissions = canOperatorPermissions(loginUser, new Object[]{id}, AuthorizationType.UDF, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_DELETE); |
|
||||||
if (!canOperatorPermissions) { |
|
||||||
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION); |
|
||||||
return result; |
|
||||||
} |
|
||||||
udfFuncMapper.deleteById(id); |
|
||||||
udfUserMapper.deleteByUdfFuncId(id); |
|
||||||
log.info("UDF function delete complete, udfFuncId:{}.", id); |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* verify udf function by name |
|
||||||
* |
|
||||||
* @param name name |
|
||||||
* @return true if the name can user, otherwise return false |
|
||||||
*/ |
|
||||||
@Override |
|
||||||
public Result<Object> verifyUdfFuncByName(User loginUser, String name) { |
|
||||||
Result<Object> result = new Result<>(); |
|
||||||
boolean canOperatorPermissions = canOperatorPermissions(loginUser, null, AuthorizationType.UDF, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW); |
|
||||||
if (!canOperatorPermissions) { |
|
||||||
putMsg(result, Status.NO_CURRENT_OPERATING_PERMISSION); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
if (checkUdfFuncNameExists(name)) { |
|
||||||
log.warn("Udf function with the same already exists."); |
|
||||||
putMsg(result, Status.UDF_FUNCTION_EXISTS); |
|
||||||
} else { |
|
||||||
putMsg(result, Status.SUCCESS); |
|
||||||
} |
|
||||||
return result; |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,94 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.api.permission; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.common.enums.AuthorizationType; |
|
||||||
import org.apache.dolphinscheduler.common.enums.UserType; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.Project; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.User; |
|
||||||
import org.apache.dolphinscheduler.dao.mapper.UdfFuncMapper; |
|
||||||
|
|
||||||
import java.util.Arrays; |
|
||||||
import java.util.Collections; |
|
||||||
import java.util.HashSet; |
|
||||||
import java.util.List; |
|
||||||
import java.util.Set; |
|
||||||
|
|
||||||
import org.junit.jupiter.api.Assertions; |
|
||||||
import org.junit.jupiter.api.Test; |
|
||||||
import org.junit.jupiter.api.extension.ExtendWith; |
|
||||||
import org.mockito.InjectMocks; |
|
||||||
import org.mockito.Mock; |
|
||||||
import org.mockito.Mockito; |
|
||||||
import org.mockito.junit.jupiter.MockitoExtension; |
|
||||||
import org.slf4j.Logger; |
|
||||||
import org.slf4j.LoggerFactory; |
|
||||||
|
|
||||||
@ExtendWith(MockitoExtension.class) |
|
||||||
public class UdfFuncPermissionCheckTest { |
|
||||||
|
|
||||||
private static final Logger logger = LoggerFactory.getLogger(UdfFuncPermissionCheckTest.class); |
|
||||||
@InjectMocks |
|
||||||
private ResourcePermissionCheckServiceImpl.UdfFuncPermissionCheck udfFuncPermissionCheck; |
|
||||||
|
|
||||||
@Mock |
|
||||||
private UdfFuncMapper udfFuncMapper; |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testPermissionCheck() { |
|
||||||
User user = getLoginUser(); |
|
||||||
Assertions.assertTrue(udfFuncPermissionCheck.permissionCheck(user.getId(), null, logger)); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testAuthorizationTypes() { |
|
||||||
List<AuthorizationType> authorizationTypes = udfFuncPermissionCheck.authorizationTypes(); |
|
||||||
Assertions.assertEquals(Collections.singletonList(AuthorizationType.UDF), authorizationTypes); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testListAuthorizedResourceIds() { |
|
||||||
User user = getLoginUser(); |
|
||||||
UdfFunc udfFunc = new UdfFunc(); |
|
||||||
Set<Integer> ids = new HashSet(); |
|
||||||
ids.add(udfFunc.getId()); |
|
||||||
List<UdfFunc> udfFuncs = Arrays.asList(udfFunc); |
|
||||||
|
|
||||||
Mockito.when(udfFuncMapper.listAuthorizedUdfByUserId(user.getId())).thenReturn(udfFuncs); |
|
||||||
|
|
||||||
Assertions.assertEquals(ids, udfFuncPermissionCheck.listAuthorizedResourceIds(user.getId(), logger)); |
|
||||||
} |
|
||||||
|
|
||||||
private User getLoginUser() { |
|
||||||
User loginUser = new User(); |
|
||||||
loginUser.setUserType(UserType.GENERAL_USER); |
|
||||||
loginUser.setUserName("test"); |
|
||||||
loginUser.setId(1); |
|
||||||
return loginUser; |
|
||||||
} |
|
||||||
|
|
||||||
private Project getProject() { |
|
||||||
Project project = new Project(); |
|
||||||
project.setCode(1L); |
|
||||||
project.setId(1); |
|
||||||
project.setName("projectName"); |
|
||||||
project.setUserId(1); |
|
||||||
return project; |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,306 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.api.service; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.api.constants.ApiFuncIdentificationConstant; |
|
||||||
import org.apache.dolphinscheduler.api.enums.Status; |
|
||||||
import org.apache.dolphinscheduler.api.permission.ResourcePermissionCheckService; |
|
||||||
import org.apache.dolphinscheduler.api.service.impl.BaseServiceImpl; |
|
||||||
import org.apache.dolphinscheduler.api.service.impl.UdfFuncServiceImpl; |
|
||||||
import org.apache.dolphinscheduler.api.utils.PageInfo; |
|
||||||
import org.apache.dolphinscheduler.api.utils.Result; |
|
||||||
import org.apache.dolphinscheduler.common.enums.AuthorizationType; |
|
||||||
import org.apache.dolphinscheduler.common.enums.UdfType; |
|
||||||
import org.apache.dolphinscheduler.common.enums.UserType; |
|
||||||
import org.apache.dolphinscheduler.common.utils.PropertyUtils; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.User; |
|
||||||
import org.apache.dolphinscheduler.dao.mapper.UDFUserMapper; |
|
||||||
import org.apache.dolphinscheduler.dao.mapper.UdfFuncMapper; |
|
||||||
import org.apache.dolphinscheduler.plugin.storage.api.StorageOperator; |
|
||||||
|
|
||||||
import org.apache.commons.collections4.CollectionUtils; |
|
||||||
|
|
||||||
import java.util.ArrayList; |
|
||||||
import java.util.Collections; |
|
||||||
import java.util.Date; |
|
||||||
import java.util.HashSet; |
|
||||||
import java.util.List; |
|
||||||
import java.util.Set; |
|
||||||
|
|
||||||
import org.junit.jupiter.api.AfterEach; |
|
||||||
import org.junit.jupiter.api.Assertions; |
|
||||||
import org.junit.jupiter.api.BeforeEach; |
|
||||||
import org.junit.jupiter.api.Test; |
|
||||||
import org.junit.jupiter.api.extension.ExtendWith; |
|
||||||
import org.mockito.InjectMocks; |
|
||||||
import org.mockito.Mock; |
|
||||||
import org.mockito.MockedStatic; |
|
||||||
import org.mockito.Mockito; |
|
||||||
import org.mockito.junit.jupiter.MockitoExtension; |
|
||||||
import org.mockito.junit.jupiter.MockitoSettings; |
|
||||||
import org.mockito.quality.Strictness; |
|
||||||
import org.slf4j.Logger; |
|
||||||
import org.slf4j.LoggerFactory; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.core.metadata.IPage; |
|
||||||
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf func service test |
|
||||||
*/ |
|
||||||
@ExtendWith(MockitoExtension.class) |
|
||||||
@MockitoSettings(strictness = Strictness.LENIENT) |
|
||||||
public class UdfFuncServiceTest { |
|
||||||
|
|
||||||
private static final Logger logger = LoggerFactory.getLogger(UdfFuncServiceTest.class); |
|
||||||
|
|
||||||
private MockedStatic<PropertyUtils> mockedStaticPropertyUtils; |
|
||||||
|
|
||||||
@InjectMocks |
|
||||||
private UdfFuncServiceImpl udfFuncService; |
|
||||||
|
|
||||||
@Mock |
|
||||||
private UdfFuncMapper udfFuncMapper; |
|
||||||
|
|
||||||
@Mock |
|
||||||
private UDFUserMapper udfUserMapper; |
|
||||||
|
|
||||||
@Mock |
|
||||||
private StorageOperator storageOperator; |
|
||||||
|
|
||||||
@BeforeEach |
|
||||||
public void setUp() { |
|
||||||
mockedStaticPropertyUtils = Mockito.mockStatic(PropertyUtils.class); |
|
||||||
} |
|
||||||
|
|
||||||
@Mock |
|
||||||
private ResourcePermissionCheckService resourcePermissionCheckService; |
|
||||||
|
|
||||||
private static final Logger serviceLogger = LoggerFactory.getLogger(BaseServiceImpl.class); |
|
||||||
private static final Logger udfLogger = LoggerFactory.getLogger(UdfFuncServiceImpl.class); |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testCreateUdfFunction() { |
|
||||||
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_CREATE, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when( |
|
||||||
resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, null, 0, serviceLogger)) |
|
||||||
.thenReturn(true); |
|
||||||
// resource not exist
|
|
||||||
Result result = udfFuncService.createUdfFunction(getLoginUser(), "UdfFuncServiceTest", |
|
||||||
"org.apache.dolphinscheduler.api.service.UdfFuncServiceTest", "String", |
|
||||||
"UdfFuncServiceTest", "UdfFuncServiceTest", "", UdfType.HIVE); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertEquals(Status.RESOURCE_NOT_EXIST.getMsg(), result.getMsg()); |
|
||||||
// success
|
|
||||||
Mockito.when(storageOperator.exists("String")).thenReturn(true); |
|
||||||
|
|
||||||
result = udfFuncService.createUdfFunction(getLoginUser(), "UdfFuncServiceTest", |
|
||||||
"org.apache.dolphinscheduler.api.service.UdfFuncServiceTest", "String", |
|
||||||
"UdfFuncServiceTest", "UdfFuncServiceTest", "", UdfType.HIVE); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertEquals(Status.SUCCESS.getMsg(), result.getMsg()); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testQueryUdfFuncDetail() { |
|
||||||
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, new Object[]{2}, 0, |
|
||||||
serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(udfFuncMapper.selectById(1)).thenReturn(getUdfFunc()); |
|
||||||
// resource not exist
|
|
||||||
Result<Object> result = udfFuncService.queryUdfFuncDetail(getLoginUser(), 2); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertTrue(Status.RESOURCE_NOT_EXIST.getCode() == result.getCode()); |
|
||||||
// success
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, new Object[]{1}, 0, |
|
||||||
serviceLogger)).thenReturn(true); |
|
||||||
result = udfFuncService.queryUdfFuncDetail(getLoginUser(), 1); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertTrue(Status.SUCCESS.getCode() == result.getCode()); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testUpdateUdfFunc() { |
|
||||||
Mockito.when(udfFuncMapper.selectUdfById(1)).thenReturn(getUdfFunc()); |
|
||||||
|
|
||||||
// UDF_FUNCTION_NOT_EXIST
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_UPDATE, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, new Object[]{12}, 0, |
|
||||||
serviceLogger)).thenReturn(true); |
|
||||||
Result<Object> result = udfFuncService.updateUdfFunc(getLoginUser(), 12, "UdfFuncServiceTest", |
|
||||||
"org.apache.dolphinscheduler.api.service.UdfFuncServiceTest", "String", |
|
||||||
"UdfFuncServiceTest", "UdfFuncServiceTest", UdfType.HIVE, ""); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertTrue(Status.UDF_FUNCTION_NOT_EXIST.getCode() == result.getCode()); |
|
||||||
|
|
||||||
// RESOURCE_NOT_EXIST
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_UPDATE, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, new Object[]{11}, 0, |
|
||||||
serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(udfFuncMapper.selectUdfById(11)).thenReturn(getUdfFunc()); |
|
||||||
result = udfFuncService.updateUdfFunc(getLoginUser(), 11, "UdfFuncServiceTest", |
|
||||||
"org.apache.dolphinscheduler.api.service.UdfFuncServiceTest", "String", |
|
||||||
"UdfFuncServiceTest", "UdfFuncServiceTest", UdfType.HIVE, ""); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertTrue(Status.RESOURCE_NOT_EXIST.getCode() == result.getCode()); |
|
||||||
|
|
||||||
// success
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_UPDATE, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(storageOperator.exists("")).thenReturn(true); |
|
||||||
|
|
||||||
result = udfFuncService.updateUdfFunc(getLoginUser(), 11, "UdfFuncServiceTest", |
|
||||||
"org.apache.dolphinscheduler.api.service.UdfFuncServiceTest", "String", |
|
||||||
"UdfFuncServiceTest", "UdfFuncServiceTest", UdfType.HIVE, ""); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertTrue(Status.SUCCESS.getCode() == result.getCode()); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testQueryUdfFuncListPaging() { |
|
||||||
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when( |
|
||||||
resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, null, 0, serviceLogger)) |
|
||||||
.thenReturn(true); |
|
||||||
Mockito.when( |
|
||||||
resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.UDF, 1, udfLogger)) |
|
||||||
.thenReturn(getSetIds()); |
|
||||||
IPage<UdfFunc> page = new Page<>(1, 10); |
|
||||||
page.setTotal(1L); |
|
||||||
page.setRecords(getList()); |
|
||||||
Mockito.when(udfFuncMapper.queryUdfFuncPaging(Mockito.any(Page.class), Mockito.anyList(), Mockito.eq("test"))) |
|
||||||
.thenReturn(page); |
|
||||||
Result result = udfFuncService.queryUdfFuncListPaging(getLoginUser(), "test", 1, 10); |
|
||||||
logger.info(result.toString()); |
|
||||||
PageInfo pageInfo = (PageInfo) result.getData(); |
|
||||||
Assertions.assertTrue(CollectionUtils.isNotEmpty(pageInfo.getTotalList())); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testQueryUdfFuncList() { |
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when( |
|
||||||
resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, null, 1, serviceLogger)) |
|
||||||
.thenReturn(true); |
|
||||||
Mockito.when( |
|
||||||
resourcePermissionCheckService.userOwnedResourceIdsAcquisition(AuthorizationType.UDF, 1, udfLogger)) |
|
||||||
.thenReturn(getSetIds()); |
|
||||||
|
|
||||||
User user = getLoginUser(); |
|
||||||
user.setUserType(UserType.GENERAL_USER); |
|
||||||
user.setId(1); |
|
||||||
Mockito.when(udfFuncMapper.getUdfFuncByType(Collections.singletonList(1), UdfType.HIVE.ordinal())) |
|
||||||
.thenReturn(getList()); |
|
||||||
Result<Object> result = udfFuncService.queryUdfFuncList(user, UdfType.HIVE.ordinal()); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertTrue(Status.SUCCESS.getCode() == result.getCode()); |
|
||||||
List<UdfFunc> udfFuncList = (List<UdfFunc>) result.getData(); |
|
||||||
Assertions.assertTrue(CollectionUtils.isNotEmpty(udfFuncList)); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testDelete() { |
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_DELETE, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when(resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, new Object[]{122}, 0, |
|
||||||
serviceLogger)).thenReturn(true); |
|
||||||
|
|
||||||
Mockito.when(udfFuncMapper.deleteById(Mockito.anyInt())).thenReturn(1); |
|
||||||
Mockito.when(udfUserMapper.deleteByUdfFuncId(Mockito.anyInt())).thenReturn(1); |
|
||||||
Result result = udfFuncService.delete(getLoginUser(), 122); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertEquals(Status.SUCCESS.getMsg(), result.getMsg()); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testVerifyUdfFuncByName() { |
|
||||||
|
|
||||||
Mockito.when(resourcePermissionCheckService.operationPermissionCheck(AuthorizationType.UDF, 1, |
|
||||||
ApiFuncIdentificationConstant.UDF_FUNCTION_VIEW, serviceLogger)).thenReturn(true); |
|
||||||
Mockito.when( |
|
||||||
resourcePermissionCheckService.resourcePermissionCheck(AuthorizationType.UDF, null, 0, serviceLogger)) |
|
||||||
.thenReturn(true); |
|
||||||
// success
|
|
||||||
Mockito.when(udfFuncMapper.queryUdfByIdStr(null, "UdfFuncServiceTest")).thenReturn(getList()); |
|
||||||
Result result = udfFuncService.verifyUdfFuncByName(getLoginUser(), "test"); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertEquals(Status.SUCCESS.getMsg(), result.getMsg()); |
|
||||||
// exist
|
|
||||||
result = udfFuncService.verifyUdfFuncByName(getLoginUser(), "UdfFuncServiceTest"); |
|
||||||
logger.info(result.toString()); |
|
||||||
Assertions.assertEquals(Status.UDF_FUNCTION_EXISTS.getMsg(), result.getMsg()); |
|
||||||
} |
|
||||||
|
|
||||||
private Set<Integer> getSetIds() { |
|
||||||
Set<Integer> set = new HashSet(); |
|
||||||
set.add(1); |
|
||||||
return set; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* create admin user |
|
||||||
* @return |
|
||||||
*/ |
|
||||||
private User getLoginUser() { |
|
||||||
|
|
||||||
User loginUser = new User(); |
|
||||||
loginUser.setUserType(UserType.ADMIN_USER); |
|
||||||
loginUser.setId(1); |
|
||||||
return loginUser; |
|
||||||
} |
|
||||||
|
|
||||||
private List<UdfFunc> getList() { |
|
||||||
List<UdfFunc> udfFuncList = new ArrayList<>(); |
|
||||||
udfFuncList.add(getUdfFunc()); |
|
||||||
return udfFuncList; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* get UdfFuncRequest id |
|
||||||
*/ |
|
||||||
private UdfFunc getUdfFunc() { |
|
||||||
UdfFunc udfFunc = new UdfFunc(); |
|
||||||
udfFunc.setFuncName("UdfFuncServiceTest"); |
|
||||||
udfFunc.setClassName("org.apache.dolphinscheduler.api.service.UdfFuncServiceTest"); |
|
||||||
udfFunc.setResourceId(0); |
|
||||||
udfFunc.setResourceName("UdfFuncServiceTest"); |
|
||||||
udfFunc.setCreateTime(new Date()); |
|
||||||
udfFunc.setDatabase("database"); |
|
||||||
udfFunc.setUpdateTime(new Date()); |
|
||||||
udfFunc.setType(UdfType.HIVE); |
|
||||||
return udfFunc; |
|
||||||
} |
|
||||||
|
|
||||||
@AfterEach |
|
||||||
public void after() { |
|
||||||
mockedStaticPropertyUtils.close(); |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,59 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.common.enums; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.annotation.EnumValue; |
|
||||||
|
|
||||||
/** |
|
||||||
* UDF type |
|
||||||
*/ |
|
||||||
public enum UdfType { |
|
||||||
|
|
||||||
/** |
|
||||||
* 0 hive; 1 spark |
|
||||||
*/ |
|
||||||
HIVE(0, "hive"), |
|
||||||
SPARK(1, "spark"); |
|
||||||
|
|
||||||
UdfType(int code, String descp) { |
|
||||||
this.code = code; |
|
||||||
this.descp = descp; |
|
||||||
} |
|
||||||
|
|
||||||
@EnumValue |
|
||||||
private final int code; |
|
||||||
private final String descp; |
|
||||||
|
|
||||||
public int getCode() { |
|
||||||
return code; |
|
||||||
} |
|
||||||
|
|
||||||
public String getDescp() { |
|
||||||
return descp; |
|
||||||
} |
|
||||||
|
|
||||||
public static UdfType of(int type) { |
|
||||||
for (UdfType ut : values()) { |
|
||||||
if (ut.getCode() == type) { |
|
||||||
return ut; |
|
||||||
} |
|
||||||
} |
|
||||||
throw new IllegalArgumentException("invalid type : " + type); |
|
||||||
} |
|
||||||
|
|
||||||
} |
|
||||||
@ -1,61 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
package org.apache.dolphinscheduler.dao.entity; |
|
||||||
|
|
||||||
import java.util.Date; |
|
||||||
|
|
||||||
import lombok.Data; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.annotation.IdType; |
|
||||||
import com.baomidou.mybatisplus.annotation.TableId; |
|
||||||
import com.baomidou.mybatisplus.annotation.TableName; |
|
||||||
|
|
||||||
@Data |
|
||||||
@TableName("t_ds_relation_udfs_user") |
|
||||||
public class UDFUser { |
|
||||||
|
|
||||||
/** |
|
||||||
* id |
|
||||||
*/ |
|
||||||
@TableId(value = "id", type = IdType.AUTO) |
|
||||||
private Integer id; |
|
||||||
|
|
||||||
/** |
|
||||||
* id |
|
||||||
*/ |
|
||||||
private int userId; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf id |
|
||||||
*/ |
|
||||||
private int udfId; |
|
||||||
|
|
||||||
/** |
|
||||||
* permission |
|
||||||
*/ |
|
||||||
private int perm; |
|
||||||
|
|
||||||
/** |
|
||||||
* create time |
|
||||||
*/ |
|
||||||
private Date createTime; |
|
||||||
|
|
||||||
/** |
|
||||||
* update time |
|
||||||
*/ |
|
||||||
private Date updateTime; |
|
||||||
} |
|
||||||
@ -1,157 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.dao.entity; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.common.enums.UdfType; |
|
||||||
import org.apache.dolphinscheduler.common.utils.JSONUtils; |
|
||||||
|
|
||||||
import java.io.IOException; |
|
||||||
import java.util.Date; |
|
||||||
import java.util.Objects; |
|
||||||
|
|
||||||
import lombok.Data; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.annotation.IdType; |
|
||||||
import com.baomidou.mybatisplus.annotation.TableField; |
|
||||||
import com.baomidou.mybatisplus.annotation.TableId; |
|
||||||
import com.baomidou.mybatisplus.annotation.TableName; |
|
||||||
import com.fasterxml.jackson.databind.DeserializationContext; |
|
||||||
import com.fasterxml.jackson.databind.KeyDeserializer; |
|
||||||
import com.google.common.base.Strings; |
|
||||||
|
|
||||||
@Data |
|
||||||
@TableName("t_ds_udfs") |
|
||||||
public class UdfFunc { |
|
||||||
|
|
||||||
/** |
|
||||||
* id |
|
||||||
*/ |
|
||||||
@TableId(value = "id", type = IdType.AUTO) |
|
||||||
private Integer id; |
|
||||||
/** |
|
||||||
* user id |
|
||||||
*/ |
|
||||||
private int userId; |
|
||||||
|
|
||||||
public String getResourceType() { |
|
||||||
return resourceType; |
|
||||||
} |
|
||||||
|
|
||||||
public void setResourceType(String resourceType) { |
|
||||||
this.resourceType = "UDF"; |
|
||||||
} |
|
||||||
|
|
||||||
@TableField(exist = false) |
|
||||||
private String resourceType = "UDF"; |
|
||||||
/** |
|
||||||
* udf function name |
|
||||||
*/ |
|
||||||
private String funcName; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf class name |
|
||||||
*/ |
|
||||||
private String className; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf argument types |
|
||||||
*/ |
|
||||||
private String argTypes; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf data base |
|
||||||
*/ |
|
||||||
private String database; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf description |
|
||||||
*/ |
|
||||||
private String description; |
|
||||||
|
|
||||||
/** |
|
||||||
* resource id |
|
||||||
*/ |
|
||||||
private int resourceId; |
|
||||||
|
|
||||||
/** |
|
||||||
* resource name |
|
||||||
*/ |
|
||||||
private String resourceName; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf function type: hive / spark |
|
||||||
*/ |
|
||||||
private UdfType type; |
|
||||||
|
|
||||||
/** |
|
||||||
* create time |
|
||||||
*/ |
|
||||||
private Date createTime; |
|
||||||
|
|
||||||
/** |
|
||||||
* update time |
|
||||||
*/ |
|
||||||
private Date updateTime; |
|
||||||
|
|
||||||
/** |
|
||||||
* user name |
|
||||||
*/ |
|
||||||
@TableField(exist = false) |
|
||||||
private String userName; |
|
||||||
|
|
||||||
@Override |
|
||||||
public boolean equals(Object o) { |
|
||||||
if (this == o) { |
|
||||||
return true; |
|
||||||
} |
|
||||||
if (o == null || getClass() != o.getClass()) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
|
|
||||||
UdfFunc udfFunc = (UdfFunc) o; |
|
||||||
|
|
||||||
if (!Objects.equals(id, udfFunc.id)) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
return !(funcName != null ? !funcName.equals(udfFunc.funcName) : udfFunc.funcName != null); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
@Override |
|
||||||
public int hashCode() { |
|
||||||
int result = id; |
|
||||||
result = 31 * result + (funcName != null ? funcName.hashCode() : 0); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
@Override |
|
||||||
public String toString() { |
|
||||||
return JSONUtils.toJsonString(this); |
|
||||||
} |
|
||||||
|
|
||||||
public static class UdfFuncDeserializer extends KeyDeserializer { |
|
||||||
|
|
||||||
@Override |
|
||||||
public Object deserializeKey(String key, DeserializationContext ctxt) throws IOException { |
|
||||||
if (Strings.isNullOrEmpty(key)) { |
|
||||||
return null; |
|
||||||
} |
|
||||||
return JSONUtils.parseObject(key, UdfFunc.class); |
|
||||||
} |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,44 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
package org.apache.dolphinscheduler.dao.mapper; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.dao.entity.UDFUser; |
|
||||||
|
|
||||||
import org.apache.ibatis.annotations.Param; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf user realtion mapper interface
|
|
||||||
*/ |
|
||||||
public interface UDFUserMapper extends BaseMapper<UDFUser> { |
|
||||||
|
|
||||||
/** |
|
||||||
* delete udf user realtion by userId |
|
||||||
* @param userId userId |
|
||||||
* @return delete result |
|
||||||
*/ |
|
||||||
int deleteByUserId(@Param("userId") int userId); |
|
||||||
|
|
||||||
/** |
|
||||||
* delete udf user realtion by function id |
|
||||||
* @param udfFuncId udfFuncId |
|
||||||
* @return delete result |
|
||||||
*/ |
|
||||||
int deleteByUdfFuncId(@Param("udfFuncId") int udfFuncId); |
|
||||||
|
|
||||||
} |
|
||||||
@ -1,118 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
package org.apache.dolphinscheduler.dao.mapper; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc; |
|
||||||
|
|
||||||
import org.apache.ibatis.annotations.Param; |
|
||||||
|
|
||||||
import java.util.List; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.core.mapper.BaseMapper; |
|
||||||
import com.baomidou.mybatisplus.core.metadata.IPage; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf function mapper interface
|
|
||||||
*/ |
|
||||||
public interface UdfFuncMapper extends BaseMapper<UdfFunc> { |
|
||||||
|
|
||||||
/** |
|
||||||
* select udf by id |
|
||||||
* @param id udf id |
|
||||||
* @return UdfFunc |
|
||||||
*/ |
|
||||||
UdfFunc selectUdfById(@Param("id") int id); |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf function by ids and function name |
|
||||||
* @param ids ids |
|
||||||
* @param funcNames funcNames |
|
||||||
* @return udf function list |
|
||||||
*/ |
|
||||||
List<UdfFunc> queryUdfByIdStr(@Param("ids") Integer[] ids, |
|
||||||
@Param("funcNames") String funcNames); |
|
||||||
|
|
||||||
/** |
|
||||||
* udf function page |
|
||||||
* @param page page |
|
||||||
* @param ids userId |
|
||||||
* @param searchVal searchVal |
|
||||||
* @return udf function IPage |
|
||||||
*/ |
|
||||||
IPage<UdfFunc> queryUdfFuncPaging(IPage<UdfFunc> page, |
|
||||||
@Param("ids") List<Integer> ids, |
|
||||||
@Param("searchVal") String searchVal); |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf function by type |
|
||||||
* @param ids userId |
|
||||||
* @param type type |
|
||||||
* @return udf function list |
|
||||||
*/ |
|
||||||
List<UdfFunc> getUdfFuncByType(@Param("ids") List<Integer> ids, |
|
||||||
@Param("type") Integer type); |
|
||||||
|
|
||||||
/** |
|
||||||
* query udf function except userId |
|
||||||
* @param userId userId |
|
||||||
* @return udf function list |
|
||||||
*/ |
|
||||||
List<UdfFunc> queryUdfFuncExceptUserId(@Param("userId") int userId); |
|
||||||
|
|
||||||
/** |
|
||||||
* query authed udf function |
|
||||||
* @param userId userId |
|
||||||
* @return udf function list |
|
||||||
*/ |
|
||||||
List<UdfFunc> queryAuthedUdfFunc(@Param("userId") int userId); |
|
||||||
|
|
||||||
/** |
|
||||||
* list authorized UDF function |
|
||||||
* @param userId userId |
|
||||||
* @param udfIds UDF function id array |
|
||||||
* @return UDF function list |
|
||||||
*/ |
|
||||||
<T> List<UdfFunc> listAuthorizedUdfFunc(@Param("userId") int userId, @Param("udfIds") T[] udfIds); |
|
||||||
|
|
||||||
/** |
|
||||||
* list UDF by resource id |
|
||||||
* @param resourceIds resource id array |
|
||||||
* @return UDF function list |
|
||||||
*/ |
|
||||||
List<UdfFunc> listUdfByResourceId(@Param("resourceIds") Integer[] resourceIds); |
|
||||||
|
|
||||||
/** |
|
||||||
* list UDF by resource fullName |
|
||||||
* @param resourceFullNames resource fullName array |
|
||||||
* @return UDF function list |
|
||||||
*/ |
|
||||||
List<UdfFunc> listUdfByResourceFullName(@Param("resourceFullNames") String[] resourceFullNames); |
|
||||||
|
|
||||||
/** |
|
||||||
* list authorized UDF by resource id |
|
||||||
* @param resourceIds resource id array |
|
||||||
* @return UDF function list |
|
||||||
*/ |
|
||||||
List<UdfFunc> listAuthorizedUdfByResourceId(@Param("userId") int userId, @Param("resourceIds") int[] resourceIds); |
|
||||||
|
|
||||||
/** |
|
||||||
* listAuthorizedUdfByUserId |
|
||||||
* @param userId |
|
||||||
* @return |
|
||||||
*/ |
|
||||||
List<UdfFunc> listAuthorizedUdfByUserId(@Param("userId") int userId); |
|
||||||
} |
|
||||||
@ -1,29 +0,0 @@ |
|||||||
<?xml version="1.0" encoding="UTF-8" ?> |
|
||||||
<!-- |
|
||||||
~ Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
~ contributor license agreements. See the NOTICE file distributed with |
|
||||||
~ this work for additional information regarding copyright ownership. |
|
||||||
~ The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
~ (the "License"); you may not use this file except in compliance with |
|
||||||
~ the License. You may obtain a copy of the License at |
|
||||||
~ |
|
||||||
~ http://www.apache.org/licenses/LICENSE-2.0 |
|
||||||
~ |
|
||||||
~ Unless required by applicable law or agreed to in writing, software |
|
||||||
~ distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
~ See the License for the specific language governing permissions and |
|
||||||
~ limitations under the License. |
|
||||||
--> |
|
||||||
|
|
||||||
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" > |
|
||||||
<mapper namespace="org.apache.dolphinscheduler.dao.mapper.UDFUserMapper"> |
|
||||||
<delete id="deleteByUserId"> |
|
||||||
delete from t_ds_relation_udfs_user |
|
||||||
where user_id = #{userId} |
|
||||||
</delete> |
|
||||||
<delete id="deleteByUdfFuncId"> |
|
||||||
delete from t_ds_relation_udfs_user |
|
||||||
where udf_id = #{udfFuncId} |
|
||||||
</delete> |
|
||||||
</mapper> |
|
||||||
@ -1,189 +0,0 @@ |
|||||||
<?xml version="1.0" encoding="UTF-8" ?> |
|
||||||
<!-- |
|
||||||
~ Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
~ contributor license agreements. See the NOTICE file distributed with |
|
||||||
~ this work for additional information regarding copyright ownership. |
|
||||||
~ The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
~ (the "License"); you may not use this file except in compliance with |
|
||||||
~ the License. You may obtain a copy of the License at |
|
||||||
~ |
|
||||||
~ http://www.apache.org/licenses/LICENSE-2.0 |
|
||||||
~ |
|
||||||
~ Unless required by applicable law or agreed to in writing, software |
|
||||||
~ distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
~ See the License for the specific language governing permissions and |
|
||||||
~ limitations under the License. |
|
||||||
--> |
|
||||||
|
|
||||||
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd" > |
|
||||||
<mapper namespace="org.apache.dolphinscheduler.dao.mapper.UdfFuncMapper"> |
|
||||||
|
|
||||||
<sql id="baseSql"> |
|
||||||
${alias}.id, ${alias}.user_id, ${alias}.func_name, ${alias}.class_name, ${alias}.type, ${alias}.arg_types, |
|
||||||
${alias}.database, ${alias}.description, ${alias}.resource_id, ${alias}.resource_name, ${alias}.create_time, ${alias}.update_time |
|
||||||
</sql> |
|
||||||
|
|
||||||
<select id="selectUdfById" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where id = #{id} |
|
||||||
</select> |
|
||||||
|
|
||||||
<select id="queryUdfByIdStr" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where 1 = 1 |
|
||||||
<if test="ids != null and ids.length > 0"> |
|
||||||
and udf.id in |
|
||||||
<foreach collection="ids" item="i" open="(" close=")" separator=","> |
|
||||||
#{i} |
|
||||||
</foreach> |
|
||||||
</if> |
|
||||||
<if test="funcNames != null and funcNames != ''"> |
|
||||||
and udf.func_name = #{funcNames} |
|
||||||
</if> |
|
||||||
order by udf.id asc |
|
||||||
</select> |
|
||||||
<select id="queryUdfFuncPaging" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
,u.user_name |
|
||||||
from t_ds_udfs udf,t_ds_user u |
|
||||||
where 1=1 and udf.user_id=u.id |
|
||||||
<if test="searchVal!= null and searchVal != ''"> |
|
||||||
and udf.func_name like concat('%', #{searchVal}, '%') |
|
||||||
</if> |
|
||||||
<if test="ids != null and ids.size() > 0"> |
|
||||||
and udf.id in |
|
||||||
<foreach collection="ids" item="i" open="(" close=")" separator=","> |
|
||||||
#{i} |
|
||||||
</foreach> |
|
||||||
</if> |
|
||||||
<!-- <if test="userId != 0">--> |
|
||||||
<!-- and udf.id in (--> |
|
||||||
<!-- select udf_id from t_ds_relation_udfs_user where user_id=#{userId}--> |
|
||||||
<!-- union select id as udf_id from t_ds_udfs where user_id=#{userId})--> |
|
||||||
<!-- </if>--> |
|
||||||
order by udf.create_time desc |
|
||||||
</select> |
|
||||||
<select id="getUdfFuncByType" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where udf.type=#{type} |
|
||||||
<if test="ids != null and ids.size() > 0"> |
|
||||||
and udf.id in |
|
||||||
<foreach collection="ids" item="i" open="(" close=")" separator=","> |
|
||||||
#{i} |
|
||||||
</foreach> |
|
||||||
</if> |
|
||||||
<!-- <if test="userId != 0">--> |
|
||||||
<!-- and udf.id in (--> |
|
||||||
<!-- select udf_id from t_ds_relation_udfs_user where user_id=#{userId}--> |
|
||||||
<!-- union select id as udf_id from t_ds_udfs where user_id=#{userId})--> |
|
||||||
<!-- </if>--> |
|
||||||
</select> |
|
||||||
<select id="queryUdfFuncExceptUserId" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where udf.user_id <![CDATA[ <> ]]> #{userId} |
|
||||||
</select> |
|
||||||
<select id="queryAuthedUdfFunc" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
SELECT |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf,t_ds_relation_udfs_user rel |
|
||||||
WHERE udf.id = rel.udf_id |
|
||||||
AND rel.user_id = #{userId} |
|
||||||
</select> |
|
||||||
<select id="listAuthorizedUdfFunc" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where |
|
||||||
udf.id in (select udf_id from t_ds_relation_udfs_user where user_id=#{userId} |
|
||||||
union select id as udf_id from t_ds_udfs where user_id=#{userId}) |
|
||||||
<if test="udfIds != null and udfIds.length > 0"> |
|
||||||
and udf.id in |
|
||||||
<foreach collection="udfIds" item="i" open="(" close=")" separator=","> |
|
||||||
#{i} |
|
||||||
</foreach> |
|
||||||
</if> |
|
||||||
</select> |
|
||||||
<select id="listUdfByResourceId" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where 1=1 |
|
||||||
<if test="resourceIds != null and resourceIds.length > 0"> |
|
||||||
and udf.resource_id in |
|
||||||
<foreach collection="resourceIds" item="i" open="(" close=")" separator=","> |
|
||||||
#{i} |
|
||||||
</foreach> |
|
||||||
</if> |
|
||||||
</select> |
|
||||||
<select id="listUdfByResourceFullName" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where 1=1 |
|
||||||
<if test="resourceFullNames != null and resourceFullNames.length > 0"> |
|
||||||
and udf.resource_name in |
|
||||||
<foreach collection="resourceFullNames" item="i" open="(" close=")" separator=","> |
|
||||||
#{i} |
|
||||||
</foreach> |
|
||||||
</if> |
|
||||||
</select> |
|
||||||
<select id="listAuthorizedUdfByResourceId" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where |
|
||||||
udf.id in (select udf_id from t_ds_relation_udfs_user where user_id=#{userId} |
|
||||||
union select id as udf_id from t_ds_udfs where user_id=#{userId}) |
|
||||||
<if test="resourceIds != null and resourceIds.length > 0"> |
|
||||||
and udf.resource_id in |
|
||||||
<foreach collection="resourceIds" item="i" open="(" close=")" separator=","> |
|
||||||
#{i} |
|
||||||
</foreach> |
|
||||||
</if> |
|
||||||
</select> |
|
||||||
|
|
||||||
<select id="listAuthorizedUdfByUserId" resultType="org.apache.dolphinscheduler.dao.entity.UdfFunc"> |
|
||||||
select |
|
||||||
<include refid="baseSql"> |
|
||||||
<property name="alias" value="udf"/> |
|
||||||
</include> |
|
||||||
from t_ds_udfs udf |
|
||||||
where 1=1 |
|
||||||
<if test="userId != 0"> |
|
||||||
and udf.id in ( |
|
||||||
select udf_id from t_ds_relation_udfs_user where user_id=#{userId} |
|
||||||
union select id as udf_id from t_ds_udfs where user_id=#{userId}) |
|
||||||
</if> |
|
||||||
</select> |
|
||||||
</mapper> |
|
||||||
|
|
||||||
@ -1,50 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.dao.entity; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc.UdfFuncDeserializer; |
|
||||||
|
|
||||||
import java.io.IOException; |
|
||||||
|
|
||||||
import org.junit.jupiter.api.Assertions; |
|
||||||
import org.junit.jupiter.api.Test; |
|
||||||
|
|
||||||
public class UdfFuncTest { |
|
||||||
|
|
||||||
/** |
|
||||||
* test UdfFuncDeserializer.deserializeKey |
|
||||||
* |
|
||||||
* @throws IOException |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testUdfFuncDeserializer() throws IOException { |
|
||||||
|
|
||||||
// UdfFuncDeserializer.deserializeKey key is null
|
|
||||||
UdfFuncDeserializer udfFuncDeserializer = new UdfFuncDeserializer(); |
|
||||||
Assertions.assertNull(udfFuncDeserializer.deserializeKey(null, null)); |
|
||||||
|
|
||||||
//
|
|
||||||
UdfFunc udfFunc = new UdfFunc(); |
|
||||||
udfFunc.setResourceName("dolphin_resource_update"); |
|
||||||
udfFunc.setResourceId(2); |
|
||||||
udfFunc.setClassName("org.apache.dolphinscheduler.test.mrUpdate"); |
|
||||||
|
|
||||||
Assertions.assertNotNull(udfFuncDeserializer.deserializeKey(udfFunc.toString(), null)); |
|
||||||
} |
|
||||||
|
|
||||||
} |
|
||||||
@ -1,184 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
package org.apache.dolphinscheduler.dao.mapper; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.common.enums.UdfType; |
|
||||||
import org.apache.dolphinscheduler.common.enums.UserType; |
|
||||||
import org.apache.dolphinscheduler.dao.BaseDaoTest; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UDFUser; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.User; |
|
||||||
|
|
||||||
import java.util.Date; |
|
||||||
import java.util.List; |
|
||||||
|
|
||||||
import org.junit.jupiter.api.Assertions; |
|
||||||
import org.junit.jupiter.api.Test; |
|
||||||
import org.springframework.beans.factory.annotation.Autowired; |
|
||||||
|
|
||||||
public class UDFUserMapperTest extends BaseDaoTest { |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UDFUserMapper udfUserMapper; |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UserMapper userMapper; |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UdfFuncMapper udfFuncMapper; |
|
||||||
|
|
||||||
/** |
|
||||||
* insert |
|
||||||
* @return UDFUser |
|
||||||
*/ |
|
||||||
private UDFUser insertOne() { |
|
||||||
UDFUser udfUser = new UDFUser(); |
|
||||||
udfUser.setUdfId(1); |
|
||||||
udfUser.setUserId(1); |
|
||||||
udfUser.setCreateTime(new Date()); |
|
||||||
udfUser.setUpdateTime(new Date()); |
|
||||||
udfUserMapper.insert(udfUser); |
|
||||||
return udfUser; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* insert UDFUser |
|
||||||
* @param user user |
|
||||||
* @param udfFunc udfFunc |
|
||||||
* @return UDFUser |
|
||||||
*/ |
|
||||||
private UDFUser insertOne(User user, UdfFunc udfFunc) { |
|
||||||
UDFUser udfUser = new UDFUser(); |
|
||||||
udfUser.setUdfId(udfFunc.getId()); |
|
||||||
udfUser.setUserId(user.getId()); |
|
||||||
udfUser.setCreateTime(new Date()); |
|
||||||
udfUser.setUpdateTime(new Date()); |
|
||||||
udfUserMapper.insert(udfUser); |
|
||||||
return udfUser; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* insert one user |
|
||||||
* @return User |
|
||||||
*/ |
|
||||||
private User insertOneUser() { |
|
||||||
User user = new User(); |
|
||||||
user.setUserName("user1"); |
|
||||||
user.setUserPassword("1"); |
|
||||||
user.setEmail("xx@123.com"); |
|
||||||
user.setUserType(UserType.GENERAL_USER); |
|
||||||
user.setCreateTime(new Date()); |
|
||||||
user.setTenantId(1); |
|
||||||
user.setQueue("dolphin"); |
|
||||||
user.setUpdateTime(new Date()); |
|
||||||
userMapper.insert(user); |
|
||||||
return user; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* insert one udf |
|
||||||
* @return UdfFunc |
|
||||||
*/ |
|
||||||
private UdfFunc insertOneUdfFunc() { |
|
||||||
UdfFunc udfFunc = new UdfFunc(); |
|
||||||
udfFunc.setFuncName("dolphin_udf_func"); |
|
||||||
udfFunc.setClassName("org.apache.dolphinscheduler.test.mr"); |
|
||||||
udfFunc.setType(UdfType.HIVE); |
|
||||||
udfFunc.setResourceId(1); |
|
||||||
udfFunc.setResourceName("dolphin_resource"); |
|
||||||
udfFunc.setCreateTime(new Date()); |
|
||||||
udfFunc.setUpdateTime(new Date()); |
|
||||||
udfFuncMapper.insert(udfFunc); |
|
||||||
return udfFunc; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test update |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testUpdate() { |
|
||||||
// insertOneUser
|
|
||||||
User user = insertOneUser(); |
|
||||||
// insertOneUdfFunc
|
|
||||||
UdfFunc udfFunc = insertOneUdfFunc(); |
|
||||||
// insertOne
|
|
||||||
UDFUser udfUser = insertOne(user, udfFunc); |
|
||||||
udfUser.setUserId(2); |
|
||||||
udfUser.setUdfId(2); |
|
||||||
int update = udfUserMapper.updateById(udfUser); |
|
||||||
Assertions.assertEquals(update, 1); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test delete |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testDelete() { |
|
||||||
// insertOneUser
|
|
||||||
User user = insertOneUser(); |
|
||||||
// insertOneUdfFunc
|
|
||||||
UdfFunc udfFunc = insertOneUdfFunc(); |
|
||||||
// insertOne
|
|
||||||
UDFUser udfUser = insertOne(user, udfFunc); |
|
||||||
int delete = udfUserMapper.deleteById(udfUser.getId()); |
|
||||||
Assertions.assertEquals(delete, 1); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test query |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testQuery() { |
|
||||||
// insertOne
|
|
||||||
UDFUser udfUser = insertOne(); |
|
||||||
// query
|
|
||||||
List<UDFUser> udfUserList = udfUserMapper.selectList(null); |
|
||||||
Assertions.assertNotEquals(0, udfUserList.size()); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test delete by userId |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testDeleteByUserId() { |
|
||||||
// insertOneUser
|
|
||||||
User user = insertOneUser(); |
|
||||||
// insertOneUdfFunc
|
|
||||||
UdfFunc udfFunc = insertOneUdfFunc(); |
|
||||||
// insertOne
|
|
||||||
UDFUser udfUser = insertOne(user, udfFunc); |
|
||||||
int delete = udfUserMapper.deleteByUserId(user.getId()); |
|
||||||
Assertions.assertEquals(1, delete); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test delete by udffuncId |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testDeleteByUdfFuncId() { |
|
||||||
// insertOneUser
|
|
||||||
User user = insertOneUser(); |
|
||||||
// insertOneUdfFunc
|
|
||||||
UdfFunc udfFunc = insertOneUdfFunc(); |
|
||||||
// insertOne
|
|
||||||
UDFUser udfUser = insertOne(user, udfFunc); |
|
||||||
int delete = udfUserMapper.deleteByUdfFuncId(udfFunc.getId()); |
|
||||||
Assertions.assertEquals(1, delete); |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,280 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
package org.apache.dolphinscheduler.dao.mapper; |
|
||||||
|
|
||||||
import static java.util.stream.Collectors.toList; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.common.enums.UdfType; |
|
||||||
import org.apache.dolphinscheduler.common.enums.UserType; |
|
||||||
import org.apache.dolphinscheduler.dao.BaseDaoTest; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UDFUser; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.UdfFunc; |
|
||||||
import org.apache.dolphinscheduler.dao.entity.User; |
|
||||||
|
|
||||||
import java.util.Arrays; |
|
||||||
import java.util.Collections; |
|
||||||
import java.util.Date; |
|
||||||
import java.util.List; |
|
||||||
|
|
||||||
import org.junit.jupiter.api.Assertions; |
|
||||||
import org.junit.jupiter.api.Test; |
|
||||||
import org.springframework.beans.factory.annotation.Autowired; |
|
||||||
|
|
||||||
import com.baomidou.mybatisplus.core.metadata.IPage; |
|
||||||
import com.baomidou.mybatisplus.extension.plugins.pagination.Page; |
|
||||||
|
|
||||||
public class UdfFuncMapperTest extends BaseDaoTest { |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UserMapper userMapper; |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UdfFuncMapper udfFuncMapper; |
|
||||||
|
|
||||||
@Autowired |
|
||||||
private UDFUserMapper udfUserMapper; |
|
||||||
|
|
||||||
/** |
|
||||||
* insert one udf |
|
||||||
* |
|
||||||
* @return UdfFunc |
|
||||||
*/ |
|
||||||
private UdfFunc insertOne(String funcName) { |
|
||||||
UdfFunc udfFunc = new UdfFunc(); |
|
||||||
udfFunc.setUserId(1); |
|
||||||
udfFunc.setFuncName(funcName); |
|
||||||
udfFunc.setClassName("org.apache.dolphinscheduler.test.mr"); |
|
||||||
udfFunc.setType(UdfType.HIVE); |
|
||||||
udfFunc.setResourceId(1); |
|
||||||
udfFunc.setResourceName("dolphin_resource"); |
|
||||||
udfFunc.setCreateTime(new Date()); |
|
||||||
udfFunc.setUpdateTime(new Date()); |
|
||||||
udfFuncMapper.insert(udfFunc); |
|
||||||
return udfFunc; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* insert one udf |
|
||||||
* |
|
||||||
* @return |
|
||||||
*/ |
|
||||||
private UdfFunc insertOne(User user) { |
|
||||||
UdfFunc udfFunc = new UdfFunc(); |
|
||||||
udfFunc.setUserId(user.getId()); |
|
||||||
udfFunc.setFuncName("dolphin_udf_func" + user.getUserName()); |
|
||||||
udfFunc.setClassName("org.apache.dolphinscheduler.test.mr"); |
|
||||||
udfFunc.setType(UdfType.HIVE); |
|
||||||
udfFunc.setResourceId(1); |
|
||||||
udfFunc.setResourceName("dolphin_resource"); |
|
||||||
udfFunc.setCreateTime(new Date()); |
|
||||||
udfFunc.setUpdateTime(new Date()); |
|
||||||
udfFuncMapper.insert(udfFunc); |
|
||||||
return udfFunc; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* insert one user |
|
||||||
* |
|
||||||
* @return User |
|
||||||
*/ |
|
||||||
private User insertOneUser() { |
|
||||||
return insertOneUser("user1"); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* insert one user |
|
||||||
* |
|
||||||
* @return User |
|
||||||
*/ |
|
||||||
private User insertOneUser(String userName) { |
|
||||||
return createGeneralUser(userName); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* create general user |
|
||||||
* |
|
||||||
* @return User |
|
||||||
*/ |
|
||||||
private User createGeneralUser(String userName) { |
|
||||||
User user = new User(); |
|
||||||
user.setUserName(userName); |
|
||||||
user.setUserPassword("1"); |
|
||||||
user.setEmail("xx@123.com"); |
|
||||||
user.setUserType(UserType.GENERAL_USER); |
|
||||||
user.setCreateTime(new Date()); |
|
||||||
user.setTenantId(1); |
|
||||||
user.setUpdateTime(new Date()); |
|
||||||
userMapper.insert(user); |
|
||||||
return user; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* insert UDFUser |
|
||||||
* |
|
||||||
* @param user user |
|
||||||
* @param udfFunc udf func |
|
||||||
* @return UDFUser |
|
||||||
*/ |
|
||||||
private UDFUser insertOneUDFUser(User user, UdfFunc udfFunc) { |
|
||||||
UDFUser udfUser = new UDFUser(); |
|
||||||
udfUser.setUdfId(udfFunc.getId()); |
|
||||||
udfUser.setUserId(user.getId()); |
|
||||||
udfUser.setCreateTime(new Date()); |
|
||||||
udfUser.setUpdateTime(new Date()); |
|
||||||
udfUserMapper.insert(udfUser); |
|
||||||
return udfUser; |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test update |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testUpdate() { |
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc = insertOne("func1"); |
|
||||||
udfFunc.setResourceName("dolphin_resource_update"); |
|
||||||
udfFunc.setResourceId(2); |
|
||||||
udfFunc.setClassName("org.apache.dolphinscheduler.test.mrUpdate"); |
|
||||||
udfFunc.setUpdateTime(new Date()); |
|
||||||
// update
|
|
||||||
int update = udfFuncMapper.updateById(udfFunc); |
|
||||||
Assertions.assertEquals(update, 1); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test delete |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testDelete() { |
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc = insertOne("func2"); |
|
||||||
// delete
|
|
||||||
int delete = udfFuncMapper.deleteById(udfFunc.getId()); |
|
||||||
Assertions.assertEquals(delete, 1); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test query udf by ids |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testQueryUdfByIdStr() { |
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc = insertOne("func3"); |
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc1 = insertOne("func4"); |
|
||||||
Integer[] idArray = new Integer[]{udfFunc.getId(), udfFunc1.getId()}; |
|
||||||
// queryUdfByIdStr
|
|
||||||
List<UdfFunc> udfFuncList = udfFuncMapper.queryUdfByIdStr(idArray, ""); |
|
||||||
Assertions.assertNotEquals(0, udfFuncList.size()); |
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test page |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testQueryUdfFuncPaging() { |
|
||||||
// insertOneUser
|
|
||||||
User user = insertOneUser(); |
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc = insertOne(user); |
|
||||||
// queryUdfFuncPaging
|
|
||||||
Page<UdfFunc> page = new Page(1, 3); |
|
||||||
|
|
||||||
IPage<UdfFunc> udfFuncIPage = |
|
||||||
udfFuncMapper.queryUdfFuncPaging(page, Collections.singletonList(udfFunc.getId()), ""); |
|
||||||
Assertions.assertNotEquals(0, udfFuncIPage.getTotal()); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test get udffunc by type |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testGetUdfFuncByType() { |
|
||||||
// insertOneUser
|
|
||||||
User user = insertOneUser(); |
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc = insertOne(user); |
|
||||||
// getUdfFuncByType
|
|
||||||
List<UdfFunc> udfFuncList = |
|
||||||
udfFuncMapper.getUdfFuncByType(Collections.singletonList(udfFunc.getId()), udfFunc.getType().ordinal()); |
|
||||||
Assertions.assertNotEquals(0, udfFuncList.size()); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test query udffunc expect userId |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testQueryUdfFuncExceptUserId() { |
|
||||||
// insertOneUser
|
|
||||||
User user1 = insertOneUser(); |
|
||||||
User user2 = insertOneUser("user2"); |
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc1 = insertOne(user1); |
|
||||||
UdfFunc udfFunc2 = insertOne(user2); |
|
||||||
List<UdfFunc> udfFuncList = udfFuncMapper.queryUdfFuncExceptUserId(user1.getId()); |
|
||||||
Assertions.assertNotEquals(0, udfFuncList.size()); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
/** |
|
||||||
* test query authed udffunc |
|
||||||
*/ |
|
||||||
@Test |
|
||||||
public void testQueryAuthedUdfFunc() { |
|
||||||
// insertOneUser
|
|
||||||
User user = insertOneUser(); |
|
||||||
|
|
||||||
// insertOne
|
|
||||||
UdfFunc udfFunc = insertOne(user); |
|
||||||
|
|
||||||
// insertOneUDFUser
|
|
||||||
UDFUser udfUser = insertOneUDFUser(user, udfFunc); |
|
||||||
// queryAuthedUdfFunc
|
|
||||||
List<UdfFunc> udfFuncList = udfFuncMapper.queryAuthedUdfFunc(user.getId()); |
|
||||||
Assertions.assertNotEquals(0, udfFuncList.size()); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
public void testListAuthorizedUdfFunc() { |
|
||||||
// create general user
|
|
||||||
User generalUser1 = createGeneralUser("user1"); |
|
||||||
User generalUser2 = createGeneralUser("user2"); |
|
||||||
|
|
||||||
// create udf function
|
|
||||||
UdfFunc udfFunc = insertOne(generalUser1); |
|
||||||
UdfFunc unauthorizdUdfFunc = insertOne(generalUser2); |
|
||||||
|
|
||||||
// udf function ids
|
|
||||||
Integer[] udfFuncIds = new Integer[]{udfFunc.getId(), unauthorizdUdfFunc.getId()}; |
|
||||||
|
|
||||||
List<UdfFunc> authorizedUdfFunc = udfFuncMapper.listAuthorizedUdfFunc(generalUser1.getId(), udfFuncIds); |
|
||||||
|
|
||||||
Assertions.assertEquals(generalUser1.getId().intValue(), udfFunc.getUserId()); |
|
||||||
Assertions.assertNotEquals(generalUser1.getId().intValue(), unauthorizdUdfFunc.getUserId()); |
|
||||||
Assertions.assertFalse(authorizedUdfFunc.stream().map(t -> t.getId()).collect(toList()) |
|
||||||
.containsAll(Arrays.asList(udfFuncIds))); |
|
||||||
|
|
||||||
// authorize object unauthorizdUdfFunc to generalUser1
|
|
||||||
insertOneUDFUser(generalUser1, unauthorizdUdfFunc); |
|
||||||
authorizedUdfFunc = udfFuncMapper.listAuthorizedUdfFunc(generalUser1.getId(), udfFuncIds); |
|
||||||
Assertions.assertTrue(authorizedUdfFunc.stream().map(t -> t.getId()).collect(toList()) |
|
||||||
.containsAll(Arrays.asList(udfFuncIds))); |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,188 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one |
|
||||||
* or more contributor license agreements. See the NOTICE file |
|
||||||
* distributed with this work for additional information |
|
||||||
* regarding copyright ownership. The ASF licenses this file |
|
||||||
* to you under the Apache License, Version 2.0 (the |
|
||||||
* "License"); you may not use this file except in compliance |
|
||||||
* with the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, |
|
||||||
* software distributed under the License is distributed on an |
|
||||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
|
||||||
* KIND, either express or implied. See the License for the |
|
||||||
* specific language governing permissions and limitations |
|
||||||
* under the License. |
|
||||||
* |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.e2e.cases; |
|
||||||
|
|
||||||
import lombok.SneakyThrows; |
|
||||||
import static org.assertj.core.api.Assertions.assertThat; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.e2e.core.Constants; |
|
||||||
import org.apache.dolphinscheduler.e2e.core.DolphinScheduler; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.LoginPage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.resource.FunctionManagePage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.resource.ResourcePage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.resource.UdfManagePage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.security.SecurityPage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.security.TenantPage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.security.UserPage; |
|
||||||
|
|
||||||
import java.io.File; |
|
||||||
import java.io.FileOutputStream; |
|
||||||
import java.io.InputStream; |
|
||||||
import java.net.URL; |
|
||||||
import java.net.URLConnection; |
|
||||||
import java.nio.file.Files; |
|
||||||
import java.nio.file.Path; |
|
||||||
import java.time.Duration; |
|
||||||
import java.util.Comparator; |
|
||||||
|
|
||||||
import org.testcontainers.shaded.org.awaitility.Awaitility; |
|
||||||
import org.junit.jupiter.api.AfterAll; |
|
||||||
import org.junit.jupiter.api.BeforeAll; |
|
||||||
import org.junit.jupiter.api.Order; |
|
||||||
import org.junit.jupiter.api.Test; |
|
||||||
import org.openqa.selenium.By; |
|
||||||
import org.openqa.selenium.WebElement; |
|
||||||
import org.openqa.selenium.remote.RemoteWebDriver; |
|
||||||
import org.openqa.selenium.support.ui.ExpectedConditions; |
|
||||||
import org.openqa.selenium.support.ui.WebDriverWait; |
|
||||||
|
|
||||||
@DolphinScheduler(composeFiles = "docker/file-manage/docker-compose.yaml") |
|
||||||
public class FunctionManageE2ETest { |
|
||||||
private static RemoteWebDriver browser; |
|
||||||
|
|
||||||
private static final String tenant = System.getProperty("user.name"); |
|
||||||
|
|
||||||
private static final String user = "admin"; |
|
||||||
|
|
||||||
private static final String password = "dolphinscheduler123"; |
|
||||||
|
|
||||||
private static final String email = "admin@gmail.com"; |
|
||||||
|
|
||||||
private static final String phone = "15800000000"; |
|
||||||
|
|
||||||
private static final String testUdfFunctionName = "test_function"; |
|
||||||
|
|
||||||
private static final String testRenameUdfFunctionName = "test_rename_function"; |
|
||||||
|
|
||||||
private static final String testUploadUdfFileName = "hive-jdbc-3.1.2.jar"; |
|
||||||
|
|
||||||
private static final String testClassName = "org.dolphinscheduler.UdfTest"; |
|
||||||
|
|
||||||
private static final String testDescription = "test_description"; |
|
||||||
|
|
||||||
private static final Path testUploadUdfFilePath = Constants.HOST_TMP_PATH.resolve(testUploadUdfFileName); |
|
||||||
|
|
||||||
@BeforeAll |
|
||||||
@SneakyThrows |
|
||||||
public static void setup() { |
|
||||||
TenantPage tenantPage = new LoginPage(browser) |
|
||||||
.login(user, password) |
|
||||||
.goToNav(SecurityPage.class) |
|
||||||
.goToTab(TenantPage.class) |
|
||||||
.create(tenant); |
|
||||||
|
|
||||||
Awaitility.await().untilAsserted(() -> assertThat(tenantPage.tenantList()) |
|
||||||
.as("Tenant list should contain newly-created tenant") |
|
||||||
.extracting(WebElement::getText) |
|
||||||
.anyMatch(it -> it.contains(tenant))); |
|
||||||
|
|
||||||
downloadFile("https://repo1.maven.org/maven2/org/apache/hive/hive-jdbc/3.1.2/hive-jdbc-3.1.2.jar", testUploadUdfFilePath.toFile().getAbsolutePath()); |
|
||||||
|
|
||||||
UserPage userPage = tenantPage.goToNav(SecurityPage.class) |
|
||||||
.goToTab(UserPage.class); |
|
||||||
|
|
||||||
new WebDriverWait(userPage.driver(), Duration.ofSeconds(20)).until(ExpectedConditions.visibilityOfElementLocated( |
|
||||||
new By.ByClassName("name"))); |
|
||||||
|
|
||||||
UdfManagePage udfManagePage = userPage.update(user, user, email, phone, tenant) |
|
||||||
.goToNav(ResourcePage.class) |
|
||||||
.goToTab(UdfManagePage.class) |
|
||||||
.uploadFile(testUploadUdfFilePath.toFile().getAbsolutePath()); |
|
||||||
|
|
||||||
udfManagePage.goToNav(ResourcePage.class) |
|
||||||
.goToTab(FunctionManagePage.class); |
|
||||||
} |
|
||||||
|
|
||||||
@AfterAll |
|
||||||
@SneakyThrows |
|
||||||
public static void cleanup() { |
|
||||||
Files.walk(Constants.HOST_CHROME_DOWNLOAD_PATH) |
|
||||||
.sorted(Comparator.reverseOrder()) |
|
||||||
.map(Path::toFile) |
|
||||||
.forEach(File::delete); |
|
||||||
|
|
||||||
Files.deleteIfExists(testUploadUdfFilePath); |
|
||||||
} |
|
||||||
|
|
||||||
static void downloadFile(String downloadUrl, String filePath) throws Exception { |
|
||||||
int byteRead; |
|
||||||
|
|
||||||
URL url = new URL(downloadUrl); |
|
||||||
|
|
||||||
URLConnection conn = url.openConnection(); |
|
||||||
InputStream inStream = conn.getInputStream(); |
|
||||||
FileOutputStream fs = new FileOutputStream(filePath); |
|
||||||
|
|
||||||
byte[] buffer = new byte[1024]; |
|
||||||
while ((byteRead = inStream.read(buffer)) != -1) { |
|
||||||
fs.write(buffer, 0, byteRead); |
|
||||||
} |
|
||||||
|
|
||||||
inStream.close(); |
|
||||||
fs.close(); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(10) |
|
||||||
void testCreateUdfFunction() { |
|
||||||
FunctionManagePage page = new FunctionManagePage(browser); |
|
||||||
|
|
||||||
page.createUdfFunction(testUdfFunctionName, testClassName, testUploadUdfFileName, testDescription); |
|
||||||
|
|
||||||
Awaitility.await().untilAsserted(() -> assertThat(page.functionList()) |
|
||||||
.as("Function list should contain newly-created file") |
|
||||||
.extracting(WebElement::getText) |
|
||||||
.anyMatch(it -> it.contains(testUdfFunctionName))); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(20) |
|
||||||
void testRenameUdfFunction() { |
|
||||||
FunctionManagePage page = new FunctionManagePage(browser); |
|
||||||
|
|
||||||
browser.navigate().refresh(); |
|
||||||
|
|
||||||
page.renameUdfFunction(testUdfFunctionName, testRenameUdfFunctionName); |
|
||||||
|
|
||||||
Awaitility.await().pollDelay(Duration.ofSeconds(2)).untilAsserted(() -> assertThat(page.functionList()) |
|
||||||
.as("Function list should contain newly-created file") |
|
||||||
.extracting(WebElement::getText) |
|
||||||
.anyMatch(it -> it.contains(testRenameUdfFunctionName))); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(30) |
|
||||||
void testDeleteUdfFunction() { |
|
||||||
FunctionManagePage page = new FunctionManagePage(browser); |
|
||||||
|
|
||||||
page.deleteUdfFunction(testRenameUdfFunctionName); |
|
||||||
|
|
||||||
Awaitility.await().untilAsserted(() -> { |
|
||||||
browser.navigate().refresh(); |
|
||||||
|
|
||||||
assertThat( |
|
||||||
page.functionList() |
|
||||||
).noneMatch( |
|
||||||
it -> it.getText().contains(testRenameUdfFunctionName) |
|
||||||
); |
|
||||||
}); |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,229 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one |
|
||||||
* or more contributor license agreements. See the NOTICE file |
|
||||||
* distributed with this work for additional information |
|
||||||
* regarding copyright ownership. The ASF licenses this file |
|
||||||
* to you under the Apache License, Version 2.0 (the |
|
||||||
* "License"); you may not use this file except in compliance |
|
||||||
* with the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, |
|
||||||
* software distributed under the License is distributed on an |
|
||||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
|
||||||
* KIND, either express or implied. See the License for the |
|
||||||
* specific language governing permissions and limitations |
|
||||||
* under the License. |
|
||||||
* |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.e2e.cases; |
|
||||||
|
|
||||||
import static org.assertj.core.api.Assertions.assertThat; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.e2e.core.Constants; |
|
||||||
import org.apache.dolphinscheduler.e2e.core.DolphinScheduler; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.LoginPage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.resource.ResourcePage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.resource.UdfManagePage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.security.SecurityPage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.security.TenantPage; |
|
||||||
import org.apache.dolphinscheduler.e2e.pages.security.UserPage; |
|
||||||
|
|
||||||
import java.io.File; |
|
||||||
import java.io.FileOutputStream; |
|
||||||
import java.io.InputStream; |
|
||||||
import java.net.URL; |
|
||||||
import java.net.URLConnection; |
|
||||||
import java.nio.file.Files; |
|
||||||
import java.nio.file.Path; |
|
||||||
import java.time.Duration; |
|
||||||
import java.util.Comparator; |
|
||||||
|
|
||||||
import org.testcontainers.shaded.org.awaitility.Awaitility; |
|
||||||
import org.junit.jupiter.api.AfterAll; |
|
||||||
import org.junit.jupiter.api.BeforeAll; |
|
||||||
import org.junit.jupiter.api.Order; |
|
||||||
import org.junit.jupiter.api.Test; |
|
||||||
import org.openqa.selenium.By; |
|
||||||
import org.openqa.selenium.WebElement; |
|
||||||
import org.openqa.selenium.remote.RemoteWebDriver; |
|
||||||
import org.openqa.selenium.support.ui.ExpectedConditions; |
|
||||||
import org.openqa.selenium.support.ui.WebDriverWait; |
|
||||||
|
|
||||||
import lombok.SneakyThrows; |
|
||||||
|
|
||||||
@DolphinScheduler(composeFiles = "docker/file-manage/docker-compose.yaml") |
|
||||||
public class UdfManageE2ETest { |
|
||||||
private static RemoteWebDriver browser; |
|
||||||
|
|
||||||
private static final String tenant = System.getProperty("user.name"); |
|
||||||
|
|
||||||
private static final String user = "admin"; |
|
||||||
|
|
||||||
private static final String password = "dolphinscheduler123"; |
|
||||||
|
|
||||||
private static final String email = "admin@gmail.com"; |
|
||||||
|
|
||||||
private static final String phone = "15800000000"; |
|
||||||
|
|
||||||
private static final String testDirectoryName = "test_directory"; |
|
||||||
|
|
||||||
private static final String testRenameDirectoryName = "test_rename_directory"; |
|
||||||
|
|
||||||
private static final String testUploadUdfFileName = "hive-jdbc-3.1.2.jar"; |
|
||||||
|
|
||||||
private static final Path testUploadUdfFilePath = Constants.HOST_TMP_PATH.resolve(testUploadUdfFileName); |
|
||||||
|
|
||||||
private static final String testUploadUdfRenameFileName = "hive-jdbc.jar"; |
|
||||||
|
|
||||||
@BeforeAll |
|
||||||
public static void setup() { |
|
||||||
TenantPage tenantPage = new LoginPage(browser) |
|
||||||
.login(user, password) |
|
||||||
.goToNav(SecurityPage.class) |
|
||||||
.goToTab(TenantPage.class) |
|
||||||
.create(tenant); |
|
||||||
|
|
||||||
Awaitility.await().untilAsserted(() -> assertThat(tenantPage.tenantList()) |
|
||||||
.as("Tenant list should contain newly-created tenant") |
|
||||||
.extracting(WebElement::getText) |
|
||||||
.anyMatch(it -> it.contains(tenant))); |
|
||||||
|
|
||||||
UserPage userPage = tenantPage.goToNav(SecurityPage.class) |
|
||||||
.goToTab(UserPage.class); |
|
||||||
|
|
||||||
new WebDriverWait(userPage.driver(), Duration.ofSeconds(20)).until(ExpectedConditions.visibilityOfElementLocated( |
|
||||||
new By.ByClassName("name"))); |
|
||||||
|
|
||||||
userPage.update(user, user, email, phone, tenant) |
|
||||||
.goToNav(ResourcePage.class) |
|
||||||
.goToTab(UdfManagePage.class); |
|
||||||
} |
|
||||||
|
|
||||||
@AfterAll |
|
||||||
@SneakyThrows |
|
||||||
public static void cleanup() { |
|
||||||
Files.walk(Constants.HOST_CHROME_DOWNLOAD_PATH) |
|
||||||
.sorted(Comparator.reverseOrder()) |
|
||||||
.map(Path::toFile) |
|
||||||
.forEach(File::delete); |
|
||||||
|
|
||||||
Files.deleteIfExists(testUploadUdfFilePath); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(10) |
|
||||||
void testCreateDirectory() { |
|
||||||
final UdfManagePage page = new UdfManagePage(browser); |
|
||||||
|
|
||||||
new WebDriverWait(page.driver(), Duration.ofSeconds(20)) |
|
||||||
.until(ExpectedConditions.urlContains("/resource-manage")); |
|
||||||
page.createDirectory(testDirectoryName); |
|
||||||
Awaitility.await().untilAsserted(() -> assertThat(page.udfList()) |
|
||||||
.as("File list should contain newly-created file") |
|
||||||
.extracting(WebElement::getText) |
|
||||||
.anyMatch(it -> it.contains(testDirectoryName))); |
|
||||||
} |
|
||||||
|
|
||||||
//when s3 the directory cannot be renamed
|
|
||||||
// @Test
|
|
||||||
// @Order(20)
|
|
||||||
// void testRenameDirectory() {
|
|
||||||
// final UdfManagePage page = new UdfManagePage(browser);
|
|
||||||
//
|
|
||||||
// page.rename(testDirectoryName, testRenameDirectoryName);
|
|
||||||
//
|
|
||||||
// await().untilAsserted(() -> {
|
|
||||||
// browser.navigate().refresh();
|
|
||||||
//
|
|
||||||
// assertThat(page.udfList())
|
|
||||||
// .as("File list should contain newly-created file")
|
|
||||||
// .extracting(WebElement::getText)
|
|
||||||
// .anyMatch(it -> it.contains(testRenameDirectoryName));
|
|
||||||
// });
|
|
||||||
// }
|
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(30) |
|
||||||
void testDeleteDirectory() { |
|
||||||
final UdfManagePage page = new UdfManagePage(browser); |
|
||||||
page.delete(testDirectoryName); |
|
||||||
|
|
||||||
Awaitility.await().untilAsserted(() -> { |
|
||||||
browser.navigate().refresh(); |
|
||||||
|
|
||||||
assertThat( |
|
||||||
page.udfList() |
|
||||||
).noneMatch( |
|
||||||
it -> it.getText().contains(testDirectoryName) |
|
||||||
); |
|
||||||
}); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(40) |
|
||||||
@SneakyThrows |
|
||||||
void testUploadUdf() { |
|
||||||
final UdfManagePage page = new UdfManagePage(browser); |
|
||||||
|
|
||||||
downloadFile("https://repo1.maven.org/maven2/org/apache/hive/hive-jdbc/3.1.2/hive-jdbc-3.1.2.jar", testUploadUdfFilePath.toFile().getAbsolutePath()); |
|
||||||
page.uploadFile(testUploadUdfFilePath.toFile().getAbsolutePath()); |
|
||||||
Awaitility.await().untilAsserted(() -> { |
|
||||||
assertThat(page.udfList()) |
|
||||||
.as("File list should contain newly-created file") |
|
||||||
.extracting(WebElement::getText) |
|
||||||
.anyMatch(it -> it.contains(testUploadUdfFileName)); |
|
||||||
}); |
|
||||||
} |
|
||||||
|
|
||||||
void downloadFile(String downloadUrl, String filePath) throws Exception { |
|
||||||
int byteRead; |
|
||||||
|
|
||||||
URL url = new URL(downloadUrl); |
|
||||||
|
|
||||||
URLConnection conn = url.openConnection(); |
|
||||||
InputStream inStream = conn.getInputStream(); |
|
||||||
FileOutputStream fs = new FileOutputStream(filePath); |
|
||||||
|
|
||||||
byte[] buffer = new byte[1024]; |
|
||||||
while ((byteRead = inStream.read(buffer)) != -1) { |
|
||||||
fs.write(buffer, 0, byteRead); |
|
||||||
} |
|
||||||
|
|
||||||
inStream.close(); |
|
||||||
fs.close(); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(60) |
|
||||||
void testRenameUdf() { |
|
||||||
final UdfManagePage page = new UdfManagePage(browser); |
|
||||||
page.rename(testUploadUdfFileName, testUploadUdfRenameFileName); |
|
||||||
|
|
||||||
Awaitility.await().untilAsserted(() -> { |
|
||||||
assertThat(page.udfList()) |
|
||||||
.as("File list should contain newly-created file") |
|
||||||
.extracting(WebElement::getText) |
|
||||||
.anyMatch(it -> it.contains(testUploadUdfRenameFileName)); |
|
||||||
}); |
|
||||||
} |
|
||||||
|
|
||||||
@Test |
|
||||||
@Order(70) |
|
||||||
void testDeleteUdf() { |
|
||||||
final UdfManagePage page = new UdfManagePage(browser); |
|
||||||
page.delete(testUploadUdfRenameFileName); |
|
||||||
|
|
||||||
Awaitility.await().untilAsserted(() -> { |
|
||||||
browser.navigate().refresh(); |
|
||||||
|
|
||||||
assertThat( |
|
||||||
page.udfList() |
|
||||||
).noneMatch( |
|
||||||
it -> it.getText().contains(testUploadUdfRenameFileName) |
|
||||||
); |
|
||||||
}); |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,201 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one |
|
||||||
* or more contributor license agreements. See the NOTICE file |
|
||||||
* distributed with this work for additional information |
|
||||||
* regarding copyright ownership. The ASF licenses this file |
|
||||||
* to you under the Apache License, Version 2.0 (the |
|
||||||
* "License"); you may not use this file except in compliance |
|
||||||
* with the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, |
|
||||||
* software distributed under the License is distributed on an |
|
||||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
|
||||||
* KIND, either express or implied. See the License for the |
|
||||||
* specific language governing permissions and limitations |
|
||||||
* under the License. |
|
||||||
* |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.e2e.pages.resource; |
|
||||||
|
|
||||||
import lombok.Getter; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.e2e.pages.common.NavBarPage; |
|
||||||
|
|
||||||
import java.util.List; |
|
||||||
|
|
||||||
import org.openqa.selenium.By; |
|
||||||
import org.openqa.selenium.JavascriptExecutor; |
|
||||||
import org.openqa.selenium.Keys; |
|
||||||
import org.openqa.selenium.WebElement; |
|
||||||
import org.openqa.selenium.remote.RemoteWebDriver; |
|
||||||
import org.openqa.selenium.support.FindBy; |
|
||||||
import org.openqa.selenium.support.FindBys; |
|
||||||
import org.openqa.selenium.support.PageFactory; |
|
||||||
import org.openqa.selenium.support.ui.ExpectedConditions; |
|
||||||
import org.openqa.selenium.support.ui.WebDriverWait; |
|
||||||
|
|
||||||
@Getter |
|
||||||
public class FunctionManagePage extends NavBarPage implements ResourcePage.Tab { |
|
||||||
@FindBy(className = "btn-create-udf-function") |
|
||||||
private WebElement buttonCreateUdfFunction; |
|
||||||
|
|
||||||
@FindBy(className = "items") |
|
||||||
private List<WebElement> functionList; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "n-popconfirm__action"), |
|
||||||
@FindBy(className = "n-button--primary-type"), |
|
||||||
}) |
|
||||||
private WebElement buttonConfirm; |
|
||||||
|
|
||||||
private CreateUdfFunctionBox createUdfFunctionBox; |
|
||||||
|
|
||||||
private RenameUdfFunctionBox renameUdfFunctionBox; |
|
||||||
|
|
||||||
public FunctionManagePage(RemoteWebDriver driver) { |
|
||||||
super(driver); |
|
||||||
|
|
||||||
createUdfFunctionBox = new CreateUdfFunctionBox(); |
|
||||||
|
|
||||||
renameUdfFunctionBox = new RenameUdfFunctionBox(); |
|
||||||
} |
|
||||||
|
|
||||||
public FunctionManagePage createUdfFunction(String udfFunctionName, String className, String udfResourceName, String description) { |
|
||||||
buttonCreateUdfFunction().click(); |
|
||||||
|
|
||||||
((JavascriptExecutor) driver).executeScript("arguments[0].click();", createUdfFunctionBox().radioFunctionType()); |
|
||||||
|
|
||||||
createUdfFunctionBox().inputFunctionName().sendKeys(udfFunctionName); |
|
||||||
|
|
||||||
createUdfFunctionBox().inputClassName().sendKeys(className); |
|
||||||
|
|
||||||
createUdfFunctionBox().inputDescription().sendKeys(description); |
|
||||||
|
|
||||||
createUdfFunctionBox().buttonUdfResourceDropDown().click(); |
|
||||||
|
|
||||||
createUdfFunctionBox().selectUdfResource() |
|
||||||
.stream() |
|
||||||
.filter(it -> it.getAttribute("innerHTML").contains(udfResourceName)) |
|
||||||
.findFirst() |
|
||||||
.orElseThrow(() -> new RuntimeException(String.format("No %s in udf resource list", udfResourceName))) |
|
||||||
.click(); |
|
||||||
|
|
||||||
createUdfFunctionBox().buttonSubmit().click(); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
public FunctionManagePage renameUdfFunction(String currentName, String afterName) { |
|
||||||
functionList() |
|
||||||
.stream() |
|
||||||
.filter(it -> it.getText().contains(currentName)) |
|
||||||
.flatMap(it -> it.findElements(By.className("btn-edit")).stream()) |
|
||||||
.filter(WebElement::isDisplayed) |
|
||||||
.findFirst() |
|
||||||
.orElseThrow(() -> new RuntimeException("No rename button in function manage list")) |
|
||||||
.click(); |
|
||||||
|
|
||||||
renameUdfFunctionBox().inputFunctionName().sendKeys(Keys.CONTROL + "a"); |
|
||||||
renameUdfFunctionBox().inputFunctionName().sendKeys(Keys.BACK_SPACE); |
|
||||||
renameUdfFunctionBox().inputFunctionName().sendKeys(afterName); |
|
||||||
|
|
||||||
renameUdfFunctionBox.buttonSubmit().click(); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
public FunctionManagePage deleteUdfFunction(String udfFunctionName) { |
|
||||||
functionList() |
|
||||||
.stream() |
|
||||||
.filter(it -> it.getText().contains(udfFunctionName)) |
|
||||||
.flatMap(it -> it.findElements(By.className("btn-delete")).stream()) |
|
||||||
.filter(WebElement::isDisplayed) |
|
||||||
.findFirst() |
|
||||||
.orElseThrow(() -> new RuntimeException("No delete button in udf resource list")) |
|
||||||
.click(); |
|
||||||
|
|
||||||
((JavascriptExecutor) driver).executeScript("arguments[0].click();", buttonConfirm()); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
@Getter |
|
||||||
public class CreateUdfFunctionBox { |
|
||||||
CreateUdfFunctionBox() { |
|
||||||
PageFactory.initElements(driver, this); |
|
||||||
} |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "radio-function-type"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement radioFunctionType; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-function-name"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement inputFunctionName; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-class-name"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement inputClassName; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "btn-udf-resource-dropdown"), |
|
||||||
@FindBy(className = "n-base-selection"), |
|
||||||
}) |
|
||||||
private WebElement buttonUdfResourceDropDown; |
|
||||||
|
|
||||||
@FindBy(className = "n-tree-node-content__text") |
|
||||||
private List<WebElement> selectUdfResource; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-description"), |
|
||||||
@FindBy(tagName = "textarea"), |
|
||||||
}) |
|
||||||
private WebElement inputDescription; |
|
||||||
|
|
||||||
@FindBy(className = "btn-submit") |
|
||||||
private WebElement buttonSubmit; |
|
||||||
|
|
||||||
@FindBy(className = "btn-cancel") |
|
||||||
private WebElement buttonCancel; |
|
||||||
} |
|
||||||
|
|
||||||
@Getter |
|
||||||
public class RenameUdfFunctionBox { |
|
||||||
RenameUdfFunctionBox() { |
|
||||||
PageFactory.initElements(driver, this); |
|
||||||
} |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-function-name"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement inputFunctionName; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-class-name"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement inputClassName; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-description"), |
|
||||||
@FindBy(tagName = "textarea"), |
|
||||||
}) |
|
||||||
private WebElement inputDescription; |
|
||||||
|
|
||||||
@FindBy(className = "btn-submit") |
|
||||||
private WebElement buttonSubmit; |
|
||||||
|
|
||||||
@FindBy(className = "btn-cancel") |
|
||||||
private WebElement buttonCancel; |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,197 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one |
|
||||||
* or more contributor license agreements. See the NOTICE file |
|
||||||
* distributed with this work for additional information |
|
||||||
* regarding copyright ownership. The ASF licenses this file |
|
||||||
* to you under the Apache License, Version 2.0 (the |
|
||||||
* "License"); you may not use this file except in compliance |
|
||||||
* with the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, |
|
||||||
* software distributed under the License is distributed on an |
|
||||||
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
|
||||||
* KIND, either express or implied. See the License for the |
|
||||||
* specific language governing permissions and limitations |
|
||||||
* under the License. |
|
||||||
* |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.e2e.pages.resource; |
|
||||||
|
|
||||||
import lombok.Getter; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.e2e.pages.common.NavBarPage; |
|
||||||
|
|
||||||
import java.time.Duration; |
|
||||||
import java.util.List; |
|
||||||
|
|
||||||
import org.openqa.selenium.By; |
|
||||||
import org.openqa.selenium.JavascriptExecutor; |
|
||||||
import org.openqa.selenium.WebElement; |
|
||||||
import org.openqa.selenium.remote.LocalFileDetector; |
|
||||||
import org.openqa.selenium.remote.RemoteWebDriver; |
|
||||||
import org.openqa.selenium.support.FindBy; |
|
||||||
import org.openqa.selenium.support.FindBys; |
|
||||||
import org.openqa.selenium.support.PageFactory; |
|
||||||
import org.openqa.selenium.support.ui.ExpectedConditions; |
|
||||||
import org.openqa.selenium.support.ui.WebDriverWait; |
|
||||||
|
|
||||||
@Getter |
|
||||||
public class UdfManagePage extends NavBarPage implements ResourcePage.Tab { |
|
||||||
@FindBy(className = "btn-create-directory") |
|
||||||
private WebElement buttonCreateDirectory; |
|
||||||
|
|
||||||
@FindBy(className = "btn-upload-resource") |
|
||||||
private WebElement buttonUploadUdf; |
|
||||||
|
|
||||||
@FindBy(className = "items") |
|
||||||
private List<WebElement> udfList; |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "n-popconfirm__action"), |
|
||||||
@FindBy(className = "n-button--primary-type"), |
|
||||||
}) |
|
||||||
private WebElement buttonConfirm; |
|
||||||
|
|
||||||
private final UploadFileBox uploadFileBox; |
|
||||||
|
|
||||||
private final RenameBox renameBox; |
|
||||||
|
|
||||||
private final CreateDirectoryBox createDirectoryBox; |
|
||||||
|
|
||||||
public UdfManagePage(RemoteWebDriver driver) { |
|
||||||
super(driver); |
|
||||||
|
|
||||||
uploadFileBox = new UploadFileBox(); |
|
||||||
|
|
||||||
renameBox = new RenameBox(); |
|
||||||
|
|
||||||
createDirectoryBox = new CreateDirectoryBox(); |
|
||||||
} |
|
||||||
|
|
||||||
public UdfManagePage createDirectory(String name) { |
|
||||||
buttonCreateDirectory().click(); |
|
||||||
|
|
||||||
createDirectoryBox().inputDirectoryName().sendKeys(name); |
|
||||||
createDirectoryBox().buttonSubmit().click(); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
public UdfManagePage uploadFile(String filePath) { |
|
||||||
new WebDriverWait(driver, Duration.ofSeconds(20)).until(ExpectedConditions.elementToBeClickable(buttonUploadUdf)); |
|
||||||
|
|
||||||
buttonUploadUdf().click(); |
|
||||||
|
|
||||||
driver.setFileDetector(new LocalFileDetector()); |
|
||||||
|
|
||||||
uploadFileBox().buttonUpload().sendKeys(filePath); |
|
||||||
uploadFileBox().buttonSubmit().click(); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
public UdfManagePage downloadFile(String fileName) { |
|
||||||
udfList() |
|
||||||
.stream() |
|
||||||
.filter(it -> it.getText().contains(fileName)) |
|
||||||
.flatMap(it -> it.findElements(By.className("btn-download")).stream()) |
|
||||||
.filter(WebElement::isDisplayed) |
|
||||||
.findFirst() |
|
||||||
.orElseThrow(() -> new RuntimeException("No download button in udf manage list")) |
|
||||||
.click(); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
public UdfManagePage rename(String currentName, String AfterName) { |
|
||||||
udfList() |
|
||||||
.stream() |
|
||||||
.filter(it -> it.getText().contains(currentName)) |
|
||||||
.flatMap(it -> it.findElements(By.className("btn-rename")).stream()) |
|
||||||
.filter(WebElement::isDisplayed) |
|
||||||
.findFirst() |
|
||||||
.orElseThrow(() -> new RuntimeException("No rename button in udf manage list")) |
|
||||||
.click(); |
|
||||||
|
|
||||||
renameBox().inputName().clear(); |
|
||||||
renameBox().inputName().sendKeys(AfterName); |
|
||||||
renameBox().buttonSubmit().click(); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
public UdfManagePage delete(String name) { |
|
||||||
udfList() |
|
||||||
.stream() |
|
||||||
.filter(it -> it.getText().contains(name)) |
|
||||||
.flatMap(it -> it.findElements(By.className("btn-delete")).stream()) |
|
||||||
.filter(WebElement::isDisplayed) |
|
||||||
.findFirst() |
|
||||||
.orElseThrow(() -> new RuntimeException("No delete button in udf manage list")) |
|
||||||
.click(); |
|
||||||
|
|
||||||
((JavascriptExecutor) driver).executeScript("arguments[0].click();", buttonConfirm()); |
|
||||||
|
|
||||||
return this; |
|
||||||
} |
|
||||||
|
|
||||||
@Getter |
|
||||||
public class RenameBox { |
|
||||||
RenameBox() { |
|
||||||
PageFactory.initElements(driver, this); |
|
||||||
} |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-name"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement inputName; |
|
||||||
|
|
||||||
@FindBy(className = "btn-submit") |
|
||||||
private WebElement buttonSubmit; |
|
||||||
|
|
||||||
@FindBy(className = "btn-cancel") |
|
||||||
private WebElement buttonCancel; |
|
||||||
} |
|
||||||
|
|
||||||
@Getter |
|
||||||
public class UploadFileBox { |
|
||||||
UploadFileBox() { |
|
||||||
PageFactory.initElements(driver, this); |
|
||||||
} |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "btn-upload"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement buttonUpload; |
|
||||||
|
|
||||||
@FindBy(className = "btn-submit") |
|
||||||
private WebElement buttonSubmit; |
|
||||||
|
|
||||||
@FindBy(className = "btn-cancel") |
|
||||||
private WebElement buttonCancel; |
|
||||||
} |
|
||||||
|
|
||||||
@Getter |
|
||||||
public class CreateDirectoryBox { |
|
||||||
CreateDirectoryBox() { |
|
||||||
PageFactory.initElements(driver, this); |
|
||||||
} |
|
||||||
|
|
||||||
@FindBys({ |
|
||||||
@FindBy(className = "input-directory-name"), |
|
||||||
@FindBy(tagName = "input"), |
|
||||||
}) |
|
||||||
private WebElement inputDirectoryName; |
|
||||||
|
|
||||||
@FindBy(className = "btn-submit") |
|
||||||
private WebElement buttonSubmit; |
|
||||||
|
|
||||||
@FindBy(className = "btn-cancel") |
|
||||||
private WebElement buttonCancel; |
|
||||||
} |
|
||||||
} |
|
||||||
@ -1,56 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.plugin.task.api.enums; |
|
||||||
|
|
||||||
/** |
|
||||||
* UDF type |
|
||||||
*/ |
|
||||||
public enum UdfType { |
|
||||||
|
|
||||||
/** |
|
||||||
* 0 hive; 1 spark |
|
||||||
*/ |
|
||||||
HIVE(0, "hive"), |
|
||||||
SPARK(1, "spark"); |
|
||||||
|
|
||||||
UdfType(int code, String descp) { |
|
||||||
this.code = code; |
|
||||||
this.descp = descp; |
|
||||||
} |
|
||||||
|
|
||||||
private final int code; |
|
||||||
private final String descp; |
|
||||||
|
|
||||||
public int getCode() { |
|
||||||
return code; |
|
||||||
} |
|
||||||
|
|
||||||
public String getDescp() { |
|
||||||
return descp; |
|
||||||
} |
|
||||||
|
|
||||||
public static UdfType of(int type) { |
|
||||||
for (UdfType ut : values()) { |
|
||||||
if (ut.getCode() == type) { |
|
||||||
return ut; |
|
||||||
} |
|
||||||
} |
|
||||||
throw new IllegalArgumentException("invalid type : " + type); |
|
||||||
} |
|
||||||
|
|
||||||
} |
|
||||||
@ -1,133 +0,0 @@ |
|||||||
/* |
|
||||||
* Licensed to the Apache Software Foundation (ASF) under one or more |
|
||||||
* contributor license agreements. See the NOTICE file distributed with |
|
||||||
* this work for additional information regarding copyright ownership. |
|
||||||
* The ASF licenses this file to You under the Apache License, Version 2.0 |
|
||||||
* (the "License"); you may not use this file except in compliance with |
|
||||||
* the License. You may obtain a copy of the License at |
|
||||||
* |
|
||||||
* http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
* |
|
||||||
* Unless required by applicable law or agreed to in writing, software |
|
||||||
* distributed under the License is distributed on an "AS IS" BASIS, |
|
||||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
|
||||||
* See the License for the specific language governing permissions and |
|
||||||
* limitations under the License. |
|
||||||
*/ |
|
||||||
|
|
||||||
package org.apache.dolphinscheduler.plugin.task.api.parameters.resource; |
|
||||||
|
|
||||||
import org.apache.dolphinscheduler.common.utils.JSONUtils; |
|
||||||
import org.apache.dolphinscheduler.plugin.task.api.enums.UdfType; |
|
||||||
|
|
||||||
import java.util.Date; |
|
||||||
|
|
||||||
import lombok.Data; |
|
||||||
|
|
||||||
import com.fasterxml.jackson.annotation.JsonProperty; |
|
||||||
|
|
||||||
@Data |
|
||||||
public class UdfFuncParameters extends AbstractResourceParameters { |
|
||||||
|
|
||||||
/** |
|
||||||
* id |
|
||||||
*/ |
|
||||||
private int id; |
|
||||||
|
|
||||||
public String getResourceType() { |
|
||||||
return resourceType; |
|
||||||
} |
|
||||||
|
|
||||||
public void setResourceType(String resourceType) { |
|
||||||
this.resourceType = resourceType; |
|
||||||
} |
|
||||||
|
|
||||||
@JsonProperty(value = "UDF") |
|
||||||
private String resourceType; |
|
||||||
|
|
||||||
/** |
|
||||||
* user id |
|
||||||
*/ |
|
||||||
private int userId; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf function name |
|
||||||
*/ |
|
||||||
private String funcName; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf class name |
|
||||||
*/ |
|
||||||
private String className; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf argument types |
|
||||||
*/ |
|
||||||
private String argTypes; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf data base |
|
||||||
*/ |
|
||||||
private String database; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf description |
|
||||||
*/ |
|
||||||
private String description; |
|
||||||
|
|
||||||
/** |
|
||||||
* resource id |
|
||||||
*/ |
|
||||||
private int resourceId; |
|
||||||
|
|
||||||
/** |
|
||||||
* resource name |
|
||||||
*/ |
|
||||||
private String resourceName; |
|
||||||
|
|
||||||
/** |
|
||||||
* udf function type: hive / spark |
|
||||||
*/ |
|
||||||
private UdfType type; |
|
||||||
|
|
||||||
/** |
|
||||||
* create time |
|
||||||
*/ |
|
||||||
private Date createTime; |
|
||||||
|
|
||||||
/** |
|
||||||
* update time |
|
||||||
*/ |
|
||||||
private Date updateTime; |
|
||||||
|
|
||||||
@Override |
|
||||||
public boolean equals(Object o) { |
|
||||||
if (this == o) { |
|
||||||
return true; |
|
||||||
} |
|
||||||
if (o == null || getClass() != o.getClass()) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
|
|
||||||
UdfFuncParameters udfFuncRequest = (UdfFuncParameters) o; |
|
||||||
|
|
||||||
if (id != udfFuncRequest.id) { |
|
||||||
return false; |
|
||||||
} |
|
||||||
return !(funcName != null ? !funcName.equals(udfFuncRequest.funcName) : udfFuncRequest.funcName != null); |
|
||||||
|
|
||||||
} |
|
||||||
|
|
||||||
@Override |
|
||||||
public int hashCode() { |
|
||||||
int result = id; |
|
||||||
result = 31 * result + (funcName != null ? funcName.hashCode() : 0); |
|
||||||
return result; |
|
||||||
} |
|
||||||
|
|
||||||
@Override |
|
||||||
public String toString() { |
|
||||||
return JSONUtils.toJsonString(this); |
|
||||||
} |
|
||||||
|
|
||||||
} |
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in new issue