Browse Source

Publish Docker images to docker hub when released (#13631)

dependabot/maven/dolphinscheduler-bom/org.yaml-snakeyaml-2.0
kezhenxu94 1 year ago committed by GitHub
parent
commit
f937c511b1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 27
      .github/workflows/publish-docker.yaml
  2. 33
      .github/workflows/publish-helm-chart.yaml
  3. 55
      docs/docs/en/contribute/release.md
  4. 51
      docs/docs/zh/contribute/release.md

27
.github/workflows/publish-docker.yaml

@ -20,9 +20,9 @@ on:
push:
branches:
- dev
env:
HUB: ghcr.io/apache/dolphinscheduler
release:
types:
- released
jobs:
build:
@ -41,12 +41,25 @@ jobs:
key: ${{ runner.os }}-maven-${{ hashFiles('**/pom.xml') }}
restore-keys: |
${{ runner.os }}-maven-
- name: Set environment variables
run: |
if [[ ${{ github.event_name }} == "release" ]]; then
echo "DOCKER_REGISTRY=docker.io" >> $GITHUB_ENV
echo "DOCKER_USERNAME=${{ secrets.DOCKERHUB_USER }}" >> $GITHUB_ENV
echo "DOCKER_PASSWORD=${{ secrets.DOCKERHUB_TOKEN }}" >> $GITHUB_ENV
echo "HUB=apache" >> $GITHUB_ENV
else
echo "DOCKER_REGISTRY=ghcr.io/apache/dolphinscheduler" >> $GITHUB_ENV
echo "DOCKER_USERNAME=${{ github.actor }}" >> $GITHUB_ENV
echo "DOCKER_PASSWORD=${{ secrets.GITHUB_TOKEN }}" >> $GITHUB_ENV
echo "HUB=ghcr.io/apache/dolphinscheduler" >> $GITHUB_ENV
fi
- name: Log in to the Container registry
uses: docker/login-action@v1.10.0
uses: docker/login-action@v2
with:
registry: ${{ env.HUB }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: ${{ env.DOCKER_REGISTRY }}
username: ${{ env.DOCKER_USERNAME }}
password: ${{ env.DOCKER_PASSWORD }}
- name: Set up QEMU
uses: docker/setup-qemu-action@v2
- name: Set up Docker Buildx

33
.github/workflows/publish-helm-chart.yaml

@ -20,6 +20,9 @@ on:
push:
branches:
- dev
release:
types:
- released
env:
HUB: ghcr.io/apache/dolphinscheduler
@ -34,17 +37,33 @@ jobs:
timeout-minutes: 30
steps:
- uses: actions/checkout@v2
- name: Set environment variables
run: |
# TODO
if [[ ${{ github.event_name }} == "release" ]]; then
echo "HUB=registry-1.docker.io/apache" >> $GITHUB_ENV
echo "DOCKER_REGISTRY=docker.io" >> $GITHUB_ENV
echo "DOCKER_USERNAME=${{ secrets.DOCKERHUB_USER }}" >> $GITHUB_ENV
echo "DOCKER_PASSWORD=${{ secrets.DOCKERHUB_TOKEN }}" >> $GITHUB_ENV
else
echo "HUB=ghcr.io/apache/dolphinscheduler" >> $GITHUB_ENV
echo "DOCKER_REGISTRY=c" >> $GITHUB_ENV
echo "DOCKER_USERNAME=${{ github.actor }}" >> $GITHUB_ENV
echo "DOCKER_PASSWORD=${{ secrets.GITHUB_TOKEN }}" >> $GITHUB_ENV
fi
- name: Log in to the Container registry
uses: docker/login-action@v1.10.0
uses: docker/login-action@v2
with:
registry: ${{ env.HUB }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
registry: ${{ env.DOCKER_REGISTRY }}
username: ${{ env.DOCKER_USERNAME }}
password: ${{ env.DOCKER_PASSWORD }}
- name: Publish Helm Chart
working-directory: deploy/kubernetes
run: |
VERSION=0.0.0-$(git rev-parse --short HEAD)
sed -i "s/^version: .*/version: $VERSION/" dolphinscheduler/Chart.yaml
if [[ ${{ env.HUB }} == "ghcr.io/apache/dolphinscheduler" ]]; then
VERSION=0.0.0-$(git rev-parse --short HEAD)
sed -i "s/^version: .*/version: $VERSION/" dolphinscheduler/Chart.yaml
fi
helm dep up dolphinscheduler
helm package dolphinscheduler
helm push dolphinscheduler-helm-$VERSION.tgz oci://${{ env.HUB }}
helm push dolphinscheduler-helm-*.tgz oci://${{ env.HUB }}

55
docs/docs/en/contribute/release.md

@ -492,56 +492,13 @@ the release version is `<VERSION>`, the following updates are required(note it w
issue template have **Version** selection bottom. So after released we should add the new `<VERSION>` to
bug-report.yml
### Publish Docker Image
### Publish Docker Image and Helm Chart
we already have the exists CI to publish the latest Docker image to GitHub container register with [config](https://github.com/apache/dolphinscheduler/blob/d80cf21456265c9d84e642bdb4db4067c7577fc6/.github/workflows/publish-docker.yaml#L55-L63).
We could reuse the main command the CI run and publish our Docker images to Docker Hub by single command.
It is highly recommended to build and test docker images locally first before push to docker hub
```shell
# Checkout and create to target tag
git checkout -b "${VERSION}" "${VERSION}"
# Build docker images locally
./mvnw -B clean package \
-Dmaven.test.skip \
-Dmaven.javadoc.skip \
-Dmaven.checkstyle.skip \
-Dmaven.deploy.skip \
-Ddocker.tag="${VERSION}" \
-Pdocker,release
# You should test whether the standalone-server images work or not
docker run --name dolphinscheduler-standalone-server -p 12345:12345 -p 25333:25333 -d apache/dolphinscheduler-standalone-server:"${DOLPHINSCHEDULER_VERSION}"
```
> Note: To push to dockerhub, you must have Apache organization permission of dockerhub. If you don’t you need to require
> from Apache infra Jira. You can refer to here to submit an application from [here](https://issues.apache.org/jira/projects/INFRA/issues/INFRA-23314)
After verifying the Docker images works as expected, you need to publish the Docker images by the following command:
```shell
./mvnw -B clean deploy \
-Dmaven.test.skip \
-Dmaven.javadoc.skip \
-Dmaven.checkstyle.skip \
-Dmaven.deploy.skip \
-Ddocker.tag="${VERSION}" \
-Ddocker.hub=apache \
-Pdocker,release
```
## Publish Helm Chart
We will also publish the Helm Chart to Docker Hub so that users don't need to download our source codes just in order
to install DolphinScheduler with Helm, run the following command to publish the Helm Chart to Docker Hub.
```bash
cd deploy/kubernetes
helm package dolphinscheduler
helm push dolphinscheduler-helm-$VERSION.tgz oci://registry-1.docker.io/apache
```
We have a [workflow](../../../../.github/workflows/publish-docker.yaml) to automatically publish Docker images
and a [workflow](../../../../.github/workflows/publish-helm-chart.yaml) to automatically publish Helm Chart to Docker Hub,
after you change the release from "pre-release" to "release", the workflow will be triggered. All you need to do
is to observe the aforementioned workflows, and after they are completed, you can pull the Docker images locally and
verify that they work as expected.
### Send Announcement E-mail Community

51
docs/docs/zh/contribute/release.md

@ -495,53 +495,10 @@ git push --delete "${GH_REMOTE}" "${VERSION}-prepare"
### 发布 Docker Image
我们已经有了现有的 CI,可以将最新的 Docker image 发布到 GitHub 容器注册 [config](https://github.com/apache/dolphinscheduler/blob/d80cf21456265c9d84e642bdb4db4067c7577fc6/.github/workflows/publish-docker.yaml#L55- L63).
我们可以重用 CI 运行的主要命令,并通过单个命令将 Docker iamge 发布到 DockerHub。
强烈建议在推送到 docker hub 之前先在本地构建和测试 docker 镜像
```shell
# Checkout and create to target tag
git checkout -b "${VERSION}" "${VERSION}"
# Build docker images locally
./mvnw -B clean package \
-Dmaven.test.skip \
-Dmaven.javadoc.skip \
-Dmaven.checkstyle.skip \
-Dmaven.deploy.skip \
-Ddocker.tag="${VERSION}" \
-Pdocker,release
# You should test whether the standalone-server images work or not
docker run --name dolphinscheduler-standalone-server -p 12345:12345 -p 25333:25333 -d apache/dolphinscheduler-standalone-server:"${DOLPHINSCHEDULER_VERSION}"
```
> 注意:推送到 dockerhub,必须有 dockerhub 的 Apache 组织权限。 如果你没有权限,你需要向 Apache infra Jira 申请。 您可以参考
> [此处](https://issues.apache.org/jira/projects/INFRA/issues/INFRA-23314)提交申请
如果您确保 docker 镜像正常工作,您需要通过以下命令将镜像发布到 Docker Hub
```shell
./mvnw -B clean deploy \
-Dmaven.test.skip \
-Dmaven.javadoc.skip \
-Dmaven.checkstyle.skip \
-Dmaven.deploy.skip \
-Ddocker.tag="${VERSION}" \
-Ddocker.hub=apache \
-Pdocker,release
```
## 发布 Helm Chart
我们也会将 Helm Chart 发布到 Docker Hub,这样用户就不需要下载我们的源码就可以使用 Helm 安装 DolphinScheduler,运行以下命令发布 Helm Chart 到 Docker Hub。
```bash
cd deploy/kubernetes
helm package dolphinscheduler
helm push dolphinscheduler-helm-$VERSION.tgz oci://registry-1.docker.io/apache
```
我们有一个 [工作流](../../../../.github/workflows/publish-docker.yaml) 来自动发布 Docker 镜像,
以及一个 [工作流](../../../../.github/workflows/publish-helm-chart.yaml) 来自动发布 Helm Chart 到 Docker Hub。
当你将发版从 "pre-release" 改为 "release" 后,这两个工作流就会被触发。你需要做的就是观察上述的工作流,
当它们完成后,你可以在本地拉取 Docker 镜像并验证它们是否按预期工作。
### 发送公告邮件通知社区

Loading…
Cancel
Save