Browse Source

[Feature-9467] add DAMENG DataSource (#12860)

* [Feature-9467][DataSource] add DAMENG (达梦) data source

* [Feature-9467] add DM datasource doc (Issue #9467)

* [Feature-9467] add DM IDataBaseOptionKeys

* [Feature-9467] add DM DataSource: DmDataSourceChannelFactoryTest remove testGetDmConnection method

* [Feature-9467] modification package dm to dameng;

* [Feature-9467] add Dameng DmJdbcDriver18 License

* [Feature-9467] add DM DataSource: remove DruidDataSourceClient

Co-authored-by: lenian <yuenian@mananacare.com>
3.2.0-release
lenian 2 years ago committed by GitHub
parent
commit
bcf03ad4d1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 22
      docs/docs/en/guide/datasource/dameng.md
  2. 17
      docs/docs/zh/guide/datasource/dameng.md
  3. BIN
      docs/img/new_ui/dev/datasource/dameng.png
  4. 7
      dolphinscheduler-bom/pom.xml
  5. 3
      dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/constants/DataSourceConstants.java
  6. 5
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-all/pom.xml
  7. 51
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/pom.xml
  8. 31
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannel.java
  9. 38
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannelFactory.java
  10. 30
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceClient.java
  11. 37
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengConnectionParam.java
  12. 42
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengDataSourceParamDTO.java
  13. 149
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengDataSourceProcessor.java
  14. 33
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/test/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannelFactoryTest.java
  15. 40
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/test/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannelTest.java
  16. 91
      dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/test/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengDataSourceProcessorTest.java
  17. 1
      dolphinscheduler-datasource-plugin/pom.xml
  18. 2
      dolphinscheduler-dist/release-docs/LICENSE
  19. 10
      dolphinscheduler-dist/release-docs/NOTICE
  20. 13
      dolphinscheduler-dist/release-docs/licenses/LICENSE-DmJdbcDriver18.txt
  21. 3
      dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java
  22. 2
      dolphinscheduler-ui/src/service/modules/data-source/types.ts
  23. 5
      dolphinscheduler-ui/src/views/datasource/list/use-form.ts
  24. 5
      dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-datasource.ts
  25. 1
      tools/dependencies/known-dependencies.txt

22
docs/docs/en/guide/datasource/dameng.md

@ -0,0 +1,22 @@
# DAMENG
![dameng](../../../../img/new_ui/dev/datasource/dameng.png)
## Datasource Parameters
| **Datasource** | **Description** |
|----------------------------|-----------------------------------------------------------|
| Datasource | Select DAMENG. |
| Datasource name | Enter the name of the DataSource. |
| Description | Enter a description of the DataSource. |
| IP/Host Name | Enter the DAMENG service IP. |
| Port | Enter the DAMENG service port. |
| Username | Set the username for DAMENG connection. |
| Password | Set the password for DAMENG connection. |
| Database name | Enter the schema name of the DAMENG connection. |
| Jdbc connection parameters | Parameter settings for DAMENG connection, in JSON format. |
## Native Supported
No, read section example in [datasource-setting](../howto/datasource-setting.md) `DataSource Center` section to activate this datasource.

17
docs/docs/zh/guide/datasource/dameng.md

@ -0,0 +1,17 @@
# DAMENG数据源
![dameng](../../../../img/new_ui/dev/datasource/dameng.png)
- 数据源:选择 DAMENG
- 数据源名称:输入数据源的名称
- 描述:输入数据源的描述
- IP 主机名:输入连接 DAMENG 的 IP
- 端口:输入连接 DAMENG 的端口
- 用户名:设置连接 DAMENG 的用户名
- 密码:设置连接 DAMENG 的密码
- 数据库名:输入连接 DAMENG 的 schema
- Jdbc 连接参数:用于 DAMENG 连接的参数设置,以 JSON 形式填写
## 是否原生支持
否,使用前需请参考 [数据源配置](../howto/datasource-setting.md) 中的 "数据源中心" 章节激活数据源。

BIN
docs/img/new_ui/dev/datasource/dameng.png

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

7
dolphinscheduler-bom/pom.xml

@ -60,6 +60,7 @@
<h2.version>2.1.210</h2.version>
<mysql-connector.version>8.0.16</mysql-connector.version>
<oracle-jdbc.version>21.5.0.0</oracle-jdbc.version>
<dameng-jdbc.version>8.1.2.79</dameng-jdbc.version>
<slf4j.version>1.7.36</slf4j.version>
<poi.version>4.1.2</poi.version>
<javax.servlet-api.version>3.1.0</javax.servlet-api.version>
@ -382,6 +383,12 @@
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.dameng</groupId>
<artifactId>DmJdbcDriver18</artifactId>
<version>${dameng-jdbc.version}</version>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>

3
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/constants/DataSourceConstants.java

@ -36,6 +36,7 @@ public class DataSourceConstants {
public static final String COM_REDSHIFT_JDBC_DRIVER = "com.amazon.redshift.jdbc42.Driver";
public static final String COM_ATHENA_JDBC_DRIVER = "com.simba.athena.jdbc.Driver";
public static final String COM_TRINO_JDBC_DRIVER = "io.trino.jdbc.TrinoDriver";
public static final String COM_DAMENG_JDBC_DRIVER = "dm.jdbc.driver.DmDriver";
/**
* validation Query
@ -51,6 +52,7 @@ public class DataSourceConstants {
public static final String REDHIFT_VALIDATION_QUERY = "select 1";
public static final String ATHENA_VALIDATION_QUERY = "select 1";
public static final String TRINO_VALIDATION_QUERY = "select 1";
public static final String DAMENG_VALIDATION_QUERY = "select 1";
/**
* jdbc url
@ -67,6 +69,7 @@ public class DataSourceConstants {
public static final String JDBC_REDSHIFT = "jdbc:redshift://";
public static final String JDBC_ATHENA = "jdbc:awsathena://";
public static final String JDBC_TRINO = "jdbc:trino://";
public static final String JDBC_DAMENG = "jdbc:dm://";
/**
* database type

5
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-all/pom.xml

@ -97,5 +97,10 @@
<artifactId>dolphinscheduler-datasource-azure-sql</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-dameng</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</project>

51
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/pom.xml

@ -0,0 +1,51 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-plugin</artifactId>
<version>dev-SNAPSHOT</version>
</parent>
<artifactId>dolphinscheduler-datasource-dameng</artifactId>
<packaging>jar</packaging>
<name>${project.artifactId}</name>
<dependencies>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-spi</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.dolphinscheduler</groupId>
<artifactId>dolphinscheduler-datasource-api</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.dameng</groupId>
<artifactId>DmJdbcDriver18</artifactId>
</dependency>
</dependencies>
</project>

31
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannel.java

@ -0,0 +1,31 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.DataSourceClient;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class DamengDataSourceChannel implements DataSourceChannel {
@Override
public DataSourceClient createDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
return new DamengDataSourceClient(baseConnectionParam, dbType);
}
}

38
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannelFactory.java

@ -0,0 +1,38 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannelFactory;
import org.apache.dolphinscheduler.spi.enums.DbType;
import com.google.auto.service.AutoService;
@AutoService(DataSourceChannelFactory.class)
public class DamengDataSourceChannelFactory implements DataSourceChannelFactory {
@Override
public String getName() {
return DbType.DAMENG.getDescp();
}
@Override
public DataSourceChannel create() {
return new DamengDataSourceChannel();
}
}

30
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceClient.java

@ -0,0 +1,30 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng;
import org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class DamengDataSourceClient extends CommonDataSourceClient {
public DamengDataSourceClient(BaseConnectionParam baseConnectionParam, DbType dbType) {
super(baseConnectionParam, dbType);
}
}

37
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengConnectionParam.java

@ -0,0 +1,37 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng.param;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
public class DamengConnectionParam extends BaseConnectionParam {
@Override
public String toString() {
return "DamengConnectionParam{"
+ "user='" + user + '\''
+ ", address='" + address + '\''
+ ", database='" + database + '\''
+ ", jdbcUrl='" + jdbcUrl + '\''
+ ", driverLocation='" + driverLocation + '\''
+ ", driverClassName='" + driverClassName + '\''
+ ", validationQuery='" + validationQuery + '\''
+ ", other='" + other + '\''
+ '}';
}
}

42
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengDataSourceParamDTO.java

@ -0,0 +1,42 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng.param;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
import org.apache.dolphinscheduler.spi.enums.DbType;
public class DamengDataSourceParamDTO extends BaseDataSourceParamDTO {
@Override
public String toString() {
return "DamengDatasourceParamDTO{"
+ "name='" + name + '\''
+ ", note='" + note + '\''
+ ", host='" + host + '\''
+ ", port=" + port
+ ", database='" + database + '\''
+ ", userName='" + userName + '\''
+ ", other='" + other + '\''
+ '}';
}
@Override
public DbType getType() {
return DbType.DAMENG;
}
}

149
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/main/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengDataSourceProcessor.java

@ -0,0 +1,149 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng.param;
import org.apache.dolphinscheduler.common.constants.Constants;
import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
import org.apache.dolphinscheduler.common.utils.JSONUtils;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.AbstractDataSourceProcessor;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.BaseDataSourceParamDTO;
import org.apache.dolphinscheduler.plugin.datasource.api.datasource.DataSourceProcessor;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.spi.datasource.BaseConnectionParam;
import org.apache.dolphinscheduler.spi.datasource.ConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.apache.commons.collections4.MapUtils;
import org.apache.commons.lang3.StringUtils;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import com.google.auto.service.AutoService;
@AutoService(DataSourceProcessor.class)
public class DamengDataSourceProcessor extends AbstractDataSourceProcessor {
private final Logger logger = LoggerFactory.getLogger(DamengDataSourceProcessor.class);
@Override
public BaseDataSourceParamDTO castDatasourceParamDTO(String paramJson) {
return JSONUtils.parseObject(paramJson, DamengDataSourceParamDTO.class);
}
@Override
public BaseDataSourceParamDTO createDatasourceParamDTO(String connectionJson) {
DamengConnectionParam connectionParams = (DamengConnectionParam) createConnectionParams(connectionJson);
DamengDataSourceParamDTO damengDatasourceParamDTO = new DamengDataSourceParamDTO();
damengDatasourceParamDTO.setUserName(connectionParams.getUser());
damengDatasourceParamDTO.setDatabase(connectionParams.getDatabase());
damengDatasourceParamDTO.setOther(connectionParams.getOther());
String address = connectionParams.getAddress();
String[] hostSeperator = address.split(Constants.DOUBLE_SLASH);
String[] hostPortArray = hostSeperator[hostSeperator.length - 1].split(Constants.COMMA);
damengDatasourceParamDTO.setPort(Integer.parseInt(hostPortArray[0].split(Constants.COLON)[1]));
damengDatasourceParamDTO.setHost(hostPortArray[0].split(Constants.COLON)[0]);
return damengDatasourceParamDTO;
}
@Override
public BaseConnectionParam createConnectionParams(BaseDataSourceParamDTO dataSourceParam) {
DamengDataSourceParamDTO dmDatasourceParam = (DamengDataSourceParamDTO) dataSourceParam;
String address = String
.format("%s%s:%s", DataSourceConstants.JDBC_DAMENG, dmDatasourceParam.getHost(),
dmDatasourceParam.getPort());
String jdbcUrl = StringUtils.isEmpty(dmDatasourceParam.getDatabase()) ? address
: String.format("%s/%s", address,
dmDatasourceParam.getDatabase());
DamengConnectionParam damengConnectionParam = new DamengConnectionParam();
damengConnectionParam.setJdbcUrl(jdbcUrl);
damengConnectionParam.setDatabase(damengConnectionParam.getDatabase());
damengConnectionParam.setAddress(address);
damengConnectionParam.setUser(dmDatasourceParam.getUserName());
damengConnectionParam.setPassword(PasswordUtils.encodePassword(dmDatasourceParam.getPassword()));
damengConnectionParam.setDriverClassName(getDatasourceDriver());
damengConnectionParam.setValidationQuery(getValidationQuery());
damengConnectionParam.setOther(dmDatasourceParam.getOther());
return damengConnectionParam;
}
@Override
public ConnectionParam createConnectionParams(String connectionJson) {
return JSONUtils.parseObject(connectionJson, DamengConnectionParam.class);
}
@Override
public String getDatasourceDriver() {
return DataSourceConstants.COM_DAMENG_JDBC_DRIVER;
}
@Override
public String getValidationQuery() {
return DataSourceConstants.DAMENG_VALIDATION_QUERY;
}
@Override
public String getJdbcUrl(ConnectionParam connectionParam) {
DamengConnectionParam damengConnectionParam = (DamengConnectionParam) connectionParam;
String jdbcUrl = damengConnectionParam.getJdbcUrl();
if (MapUtils.isNotEmpty(damengConnectionParam.getOther())) {
return String.format("%s?%s", jdbcUrl, transformOther(damengConnectionParam.getOther()));
}
return jdbcUrl;
}
@Override
public Connection getConnection(ConnectionParam connectionParam) throws ClassNotFoundException, SQLException {
DamengConnectionParam damengConnectionParam = (DamengConnectionParam) connectionParam;
Class.forName(getDatasourceDriver());
return DriverManager.getConnection(getJdbcUrl(damengConnectionParam), damengConnectionParam.getUser(),
PasswordUtils.decodePassword(damengConnectionParam.getPassword()));
}
@Override
public DbType getDbType() {
return DbType.DAMENG;
}
@Override
public DataSourceProcessor create() {
return new DamengDataSourceProcessor();
}
private String transformOther(Map<String, String> paramMap) {
if (MapUtils.isEmpty(paramMap)) {
return null;
}
List<String> otherList = new ArrayList<>();
paramMap.forEach((key, value) -> otherList.add(String.format("%s=%s", key, value)));
return String.join("&", otherList);
}
}

33
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/test/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannelFactoryTest.java

@ -0,0 +1,33 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng;
import org.apache.dolphinscheduler.spi.datasource.DataSourceChannel;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
public class DamengDataSourceChannelFactoryTest {
@Test
public void testCreate() {
DamengDataSourceChannelFactory sourceChannelFactory = new DamengDataSourceChannelFactory();
DataSourceChannel dataSourceChannel = sourceChannelFactory.create();
Assertions.assertNotNull(dataSourceChannel);
}
}

40
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/test/java/org/apache/dolphinscheduler/plugin/datasource/dameng/DamengDataSourceChannelTest.java

@ -0,0 +1,40 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng;
import org.apache.dolphinscheduler.plugin.datasource.dameng.param.DamengConnectionParam;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
@ExtendWith(MockitoExtension.class)
public class DamengDataSourceChannelTest {
@Test
public void testCreateDataSourceClient() {
DamengDataSourceChannel sourceChannel = Mockito.mock(DamengDataSourceChannel.class);
DamengDataSourceClient dataSourceClient = Mockito.mock(DamengDataSourceClient.class);
Mockito.when(sourceChannel.createDataSourceClient(Mockito.any(), Mockito.any())).thenReturn(dataSourceClient);
Assertions.assertNotNull(sourceChannel.createDataSourceClient(new DamengConnectionParam(), DbType.DAMENG));
}
}

91
dolphinscheduler-datasource-plugin/dolphinscheduler-datasource-dameng/src/test/java/org/apache/dolphinscheduler/plugin/datasource/dameng/param/DamengDataSourceProcessorTest.java

@ -0,0 +1,91 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.dolphinscheduler.plugin.datasource.dameng.param;
import org.apache.dolphinscheduler.common.constants.DataSourceConstants;
import org.apache.dolphinscheduler.plugin.datasource.api.utils.PasswordUtils;
import org.apache.dolphinscheduler.spi.enums.DbType;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.MockedStatic;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;
@ExtendWith(MockitoExtension.class)
public class DamengDataSourceProcessorTest {
private DamengDataSourceProcessor damengDatasourceProcessor = new DamengDataSourceProcessor();
@Test
public void testCreateConnectionParams() {
DamengDataSourceParamDTO damengDatasourceParamDTO = new DamengDataSourceParamDTO();
damengDatasourceParamDTO.setUserName("SYSDBA");
damengDatasourceParamDTO.setPassword("SYSDBA");
damengDatasourceParamDTO.setHost("localhost");
damengDatasourceParamDTO.setPort(5236);
damengDatasourceParamDTO.setDatabase("PERSON");
try (MockedStatic<PasswordUtils> mockedPasswordUtils = Mockito.mockStatic(PasswordUtils.class)) {
Mockito.when(PasswordUtils.encodePassword(Mockito.anyString())).thenReturn("test");
DamengConnectionParam connectionParams = (DamengConnectionParam) damengDatasourceProcessor
.createConnectionParams(damengDatasourceParamDTO);
Assertions.assertEquals("jdbc:dm://localhost:5236", connectionParams.getAddress());
Assertions.assertEquals("jdbc:dm://localhost:5236/PERSON", connectionParams.getJdbcUrl());
}
}
@Test
public void testCreateConnectionParams2() {
String connectionJson = "{\"user\":\"SYSDBA\",\"password\":\"SYSDBA\","
+ "\"address\":\"jdbc:dm://localhost:5236\""
+ ",\"database\":\"PERSON\",\"jdbcUrl\":\"jdbc:dm://localhost:5236/PERSON\"}";
DamengConnectionParam connectionParams = (DamengConnectionParam) damengDatasourceProcessor
.createConnectionParams(connectionJson);
Assertions.assertNotNull(connectionJson);
Assertions.assertEquals("SYSDBA", connectionParams.getUser());
}
@Test
public void testGetDatasourceDriver() {
Assertions.assertEquals(DataSourceConstants.COM_DAMENG_JDBC_DRIVER,
damengDatasourceProcessor.getDatasourceDriver());
}
@Test
public void testGetJdbcUrl() {
DamengConnectionParam damengConnectionParam = new DamengConnectionParam();
damengConnectionParam.setJdbcUrl("jdbc:dm://localhost:5236/PERSON");
Assertions.assertEquals(
"jdbc:dm://localhost:5236/PERSON",
damengDatasourceProcessor.getJdbcUrl(damengConnectionParam));
}
@Test
public void testGetDbType() {
Assertions.assertEquals(DbType.DAMENG, damengDatasourceProcessor.getDbType());
}
@Test
public void testGetValidationQuery() {
Assertions.assertEquals(DataSourceConstants.DAMENG_VALIDATION_QUERY,
damengDatasourceProcessor.getValidationQuery());
}
}

1
dolphinscheduler-datasource-plugin/pom.xml

@ -44,6 +44,7 @@
<module>dolphinscheduler-datasource-trino</module>
<module>dolphinscheduler-datasource-starrocks</module>
<module>dolphinscheduler-datasource-azure-sql</module>
<module>dolphinscheduler-datasource-dameng</module>
</modules>
<dependencyManagement>

2
dolphinscheduler-dist/release-docs/LICENSE vendored

@ -516,7 +516,7 @@ The text of each license is also included at licenses/LICENSE-[project].txt.
reactor-core 3.4.22: https://mvnrepository.com/artifact/io.projectreactor/reactor-core/3.4.22, Apache 2.0
reactor-netty-core 1.0.22: https://mvnrepository.com/artifact/io.projectreactor.netty/reactor-netty-core/1.0.22, Apache 2.0
reactor-netty-http 1.0.22: https://mvnrepository.com/artifact/io.projectreactor.netty/reactor-netty-http/1.0.22, Apache 2.0
DmJdbcDriver18 8.1.2.79: https://mvnrepository.com/artifact/com.dameng/DmJdbcDriver18/8.1.2.79, Apache 2.0
jna-platform

10
dolphinscheduler-dist/release-docs/NOTICE vendored

@ -2096,3 +2096,13 @@ perfmark NOTICE
* agent/src/main/resources/io/perfmark/agent/third_party/asm/LICENSE (BSD style License)
* HOMEPAGE:
* https://asm.ow2.io/
========================================================================
Dameng DmJdbcDriver18 NOTICE
========================================================================
Dameng DmJdbcDriver18
Copyright 2000-2022 Dameng Group Holding Ltd.

13
dolphinscheduler-dist/release-docs/licenses/LICENSE-DmJdbcDriver18.txt vendored

@ -0,0 +1,13 @@
Copyright 2000-2022 Dameng Group Holding Ltd.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

3
dolphinscheduler-spi/src/main/java/org/apache/dolphinscheduler/spi/enums/DbType.java

@ -42,7 +42,8 @@ public enum DbType {
ATHENA(11, "athena"),
TRINO(12, "trino"),
STARROCKS(13, "starrocks"),
AZURESQL(14, "azuresql");
AZURESQL(14, "azuresql"),
DAMENG(15, "dameng");
private static final Map<Integer, DbType> DB_TYPE_MAP =
Arrays.stream(DbType.values()).collect(toMap(DbType::getCode, Functions.identity()));

2
dolphinscheduler-ui/src/service/modules/data-source/types.ts

@ -30,6 +30,7 @@ type IDataBase =
| 'TRINO'
| 'AZURESQL'
| 'STARROCKS'
| 'DAMENG'
type IDataBaseLabel =
| 'MYSQL'
@ -46,6 +47,7 @@ type IDataBaseLabel =
| 'TRINO'
| 'AZURESQL'
| 'STARROCKS'
| 'DAMENG'
interface IDataSource {
id?: number

5
dolphinscheduler-ui/src/views/datasource/list/use-form.ts

@ -367,6 +367,11 @@ export const datasourceType: IDataBaseOptionKeys = {
value: 'STARROCKS',
label: 'STARROCKS',
defaultPort: 9030
},
DAMENG: {
value: 'DAMENG',
label: 'DAMENG',
defaultPort: 5236
}
}

5
dolphinscheduler-ui/src/views/projects/task/components/node/fields/use-datasource.ts

@ -108,6 +108,11 @@ export function useDatasource(
code: 'AZURESQL',
disabled: false
},
{
id: 15,
code: 'DAMENG',
disabled: false
}
]
const getDatasourceTypes = async () => {

1
tools/dependencies/known-dependencies.txt

@ -398,3 +398,4 @@ oauth2-oidc-sdk-9.35.jar
reactor-core-3.4.22.jar
reactor-netty-core-1.0.22.jar
reactor-netty-http-1.0.22.jar
DmJdbcDriver18-8.1.2.79.jar

Loading…
Cancel
Save