Browse Source

[fix][data quality] fix when running data quality task with Error "No main class set in JAR" (#13360)

3.2.0-release
v-wx-v 2 years ago committed by GitHub
parent
commit
e39bf59b4b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 2
      docs/docs/en/guide/data-quality.md
  2. 14
      docs/docs/zh/guide/data-quality.md
  3. 15
      dolphinscheduler-data-quality/pom.xml

2
docs/docs/en/guide/data-quality.md

@ -22,7 +22,7 @@ data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar
- Please fill in `data-quality.jar.name` according to the actual package name.
- If you package `data-quality` separately, remember to modify the package name to be consistent with `data-quality.jar.name`.
- If the old version is upgraded and used, you need to execute the `sql` update script to initialize the database before running.
- If you want to use `MySQL` data, you need to comment out the `scope` of `MySQL` in `pom.xml`.
- `dolphinscheduler-data-quality-dev-SNAPSHOT.jar` was built with no dependencies. If a `JDBC` driver is required, you can set the `-jars` parameter in the `node settings` `Option Parameters`, e.g. `--jars /lib/jars/mysql-connector-java-8.0.16.jar`.
- Currently only `MySQL`, `PostgreSQL` and `HIVE` data sources have been tested, other data sources have not been tested yet.
- `Spark` needs to be configured to read `Hive` metadata, `Spark` does not use `jdbc` to read `Hive`.

14
docs/docs/zh/guide/data-quality.md

@ -19,12 +19,14 @@
data-quality.jar.name=dolphinscheduler-data-quality-dev-SNAPSHOT.jar
```
这里的`data-quality.jar.name`请根据实际打包的名称来填写,
如果单独打包`data-quality`的话,记得修改包名和`data-quality.jar.name`一致。
如果是老版本升级使用,运行之前需要先执行`sql`更新脚本进行数据库初始化。
如果要用到`MySQL`数据,需要将`pom.xml`中`MySQL`的`scope`注释掉
当前只测试了`MySQL`、`PostgreSQL`和`HIVE`数据源,其他数据源暂时未测试过
`Spark`需要配置好读取`Hive`元数据,`Spark`不是采用`jdbc`的方式读取`Hive`
- 这里的`data-quality.jar.name`请根据实际打包的名称来填写。
- 如果单独打包`data-quality`的话,记得修改包名和`data-quality.jar.name`一致。
- 如果是老版本升级使用,运行之前需要先执行`SQL`更新脚本进行数据库初始化。
- 当前 `dolphinscheduler-data-quality-dev-SNAPSHOT.jar` 是瘦包,不包含任何 `JDBC` 驱动。
如果有 `JDBC` 驱动需要,可以在`节点设置` `选项参数`处设置 `--jars` 参数,
如:`--jars /lib/jars/mysql-connector-java-8.0.16.jar`。
- 当前只测试了`MySQL`、`PostgreSQL`和`HIVE`数据源,其他数据源暂时未测试过。
- `Spark`需要配置好读取`Hive`元数据,`Spark`不是采用`JDBC`的方式读取`Hive`。
## 检查逻辑详解

15
dolphinscheduler-data-quality/pom.xml

@ -178,4 +178,19 @@
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<mainClass>org.apache.dolphinscheduler.data.quality.DataQualityApplication</mainClass>
</manifest>
</archive>
</configuration>
</plugin>
</plugins>
</build>
</project>

Loading…
Cancel
Save