Currently, our Python API code is a module in apache/dolphinscheduler codebase,
each time users change Python API code, they need to run all requests CI check
for dolphinscheduler and Python API, But if the user does only change Python
code, it could be merged if Python API CI pass and do not dependent on others CI.
Besides, we release Python API as the same version of dolphinscheduler. It is
easy for user to match Python API version. But when Python API does not change
any code, but dolphinscheduler release a bugfix version, Python API has to
release the new version to match dolphinscheduler. This happened when we
released Python API 2.0.6 and 2.0.7. 2.0.6 and 2.0.7 is bugfix version, and
Python API does not change any code, so the PyPI package is the same.
Separate Python API also makes our code more sense, we will have more
distinguished code in dolphinscheduler and Python API new repository.
Have separate issue tracker and changelog for information to users.
ref PR in other repository: apache/dolphinscheduler-sdk-python#1
see more detail in mail thread: https://lists.apache.org/thread/4z7l5l54c4d81smjlk1n8nq380p9f0oo
* Provide aop way as an optional way to collect yarn job's applicationId, and import new module `dolphinscheduler-aop` to place the aop code.
* Add user property `appId.collect` for user to decide how to collect applicationId.
* Add new environment configuration for each type of yarn tasks to support aop in `dolphinscheduler_env.sh`
* Update docs to declare how to use aop way.
* Update `LogUtils` to support fetch applicationId in different ways based on the user property.
Co-authored-by: gabrywu <gabrywu@apache.com>
To make our release process easier, I plan to mark milestone
and label is required before the PR merged, they use for
* label: In this path we required label with at least on with
'feature|bug|improvement|document|chore' to make our changelog
more convenient
* milestone: For easier to cherry-pick PR, currently some of
our milestone only exists in issue and not exists in related
PR, it may
* [fix] Database change not sync
* Make del sql upgrade same as release version
* Correct data quality dml sql
* Upgrade database DDL to avoid log path too long error
* Add missing if exists
* Using sphinx-multiversion to support multiple version
of python api document
* change python api sidebar link
* fix whitelist_externals deprecated and add git
close: #8227
When I try to release the 3.0.0 on branch `3.0.0-prepare`,
I find out cluster-test error due to version change of
tarball, I think we should make cluster test work on both
dev and prepare release branch, so I add this patch
* when task with more than one upstreams, mapper
TaskDefinitionMapper method queryByName will return
more than one record, and failed the mybatis result
type, so we have to add `limit 1` to it to
* add multiple runs of example in integrate test
* Change from subprocess.Popen to subprocess.call_check
in integrating test which will raise an error when failed
this patch change python integrate test strategy,
before that, only content change in dir
dolphinscheduler-python will run the test. And it
will fail our python api startter when some code
change in java gateway. So we should start the
integrate test except docs change.
and in the future, we may add python ci to
.asf.yaml to make sure it pass
close: #10488
mergable some time will become crazy and adding
or deleting labels in the PR. This affects the
user experience. I try to solve it in #7099 but
it is seems not affect and we should remove the
label and comment operation and only should the
result in check result.
Use related path in our docs for imgs, previous
can not use because website need the absolute path
from root directory, and after we merged
apache/dolphinscheduler-website#789 we have covert
function to do that
close: #9426
Eric Gao
Jay Chung
Aaron Wang
kezhenxu94
caishunfeng
Kerwin
rickchengx
Daniel Y
xiangzihao
Jiajie Zhong
Wenjun Ruan
Devosend
qianli2022
Kirs