Browse Source

[python] Correct some docs for python api (#10772)

* Remove add leading `$` from command
* Add copy button from code block
* Using `python -m pip` to install package avoid error
3.1.0-release
Jiajie Zhong 2 years ago committed by GitHub
parent
commit
b4eee7edb6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
  1. 4
      dolphinscheduler-python/pydolphinscheduler/DEVELOP.md
  2. 6
      dolphinscheduler-python/pydolphinscheduler/README.md
  3. 1
      dolphinscheduler-python/pydolphinscheduler/docs/source/conf.py
  4. 71
      dolphinscheduler-python/pydolphinscheduler/docs/source/config.rst
  5. 6
      dolphinscheduler-python/pydolphinscheduler/docs/source/howto/remote-submit.rst
  6. 49
      dolphinscheduler-python/pydolphinscheduler/docs/source/start.rst
  7. 1
      dolphinscheduler-python/pydolphinscheduler/setup.py

4
dolphinscheduler-python/pydolphinscheduler/DEVELOP.md

@ -34,7 +34,7 @@ Now, we should install all dependence to make sure we could run test or check co
```shell ```shell
cd dolphinscheduler/dolphinscheduler-python/pydolphinscheduler cd dolphinscheduler/dolphinscheduler-python/pydolphinscheduler
pip install .[dev] python -m pip install .[dev]
``` ```
Next, we have to open pydolphinscheduler project in you editor. We recommend you use [pycharm][pycharm] Next, we have to open pydolphinscheduler project in you editor. We recommend you use [pycharm][pycharm]
@ -154,7 +154,7 @@ GitHub. You may locally ensure docs could be built suceessfully in case the fail
To build docs locally, install sphinx and related python modules first via: To build docs locally, install sphinx and related python modules first via:
```shell ```shell
pip install '.[doc]' python -m pip install '.[doc]'
``` ```
Then Then

6
dolphinscheduler-python/pydolphinscheduler/README.md

@ -38,11 +38,11 @@ your workflow by python code, aka workflow-as-codes.
```shell ```shell
# Install # Install
$ pip install apache-dolphinscheduler python -m pip install apache-dolphinscheduler
# Verify installation is successful, it will show the version of apache-dolphinscheduler, here we use 0.1.0 as example # Verify installation is successful, it will show the version of apache-dolphinscheduler, here we use 0.1.0 as example
$ pydolphinscheduler version pydolphinscheduler version
0.1.0 # 0.1.0
``` ```
Here we show you how to install and run a simple example of pydolphinscheduler Here we show you how to install and run a simple example of pydolphinscheduler

1
dolphinscheduler-python/pydolphinscheduler/docs/source/conf.py

@ -59,6 +59,7 @@ extensions = [
"sphinx_click.ext", "sphinx_click.ext",
# Add inline tabbed content # Add inline tabbed content
"sphinx_inline_tabs", "sphinx_inline_tabs",
"sphinx_copybutton",
] ]
# Add any paths that contain templates here, relative to this directory. # Add any paths that contain templates here, relative to this directory.

71
dolphinscheduler-python/pydolphinscheduler/docs/source/config.rst

@ -47,10 +47,10 @@ how to change them by Bash.
.. code-block:: bash .. code-block:: bash
# Modify Java Gateway Address # Modify Java Gateway Address
$ export PYDS_JAVA_GATEWAY_ADDRESS="192.168.1.1" export PYDS_JAVA_GATEWAY_ADDRESS="192.168.1.1"
# Modify Workflow Default User # Modify Workflow Default User
$ export PYDS_WORKFLOW_USER="custom-user" export PYDS_WORKFLOW_USER="custom-user"
After executing the commands above, both ``PYDS_JAVA_GATEWAY_ADDRESS`` and ``PYDS_WORKFLOW_USER`` will be changed. After executing the commands above, both ``PYDS_JAVA_GATEWAY_ADDRESS`` and ``PYDS_WORKFLOW_USER`` will be changed.
The next time you execute and submit your workflow, it will submit to host `192.168.1.1`, and with workflow's user The next time you execute and submit your workflow, it will submit to host `192.168.1.1`, and with workflow's user
@ -136,15 +136,15 @@ integrated built-in configurations in its package, but you could also export it
.. code-block:: bash .. code-block:: bash
$ pydolphinscheduler config --init pydolphinscheduler config --init
And it will create a new YAML file in the path `~/pydolphinscheduler/config.yaml` by default. If you want to export And it will create a new YAML file in the path `~/pydolphinscheduler/config.yaml` by default. If you want to export
it to another path, you should set `PYDS_HOME` before you run command :code:`pydolphinscheduler config --init`. it to another path, you should set `PYDS_HOME` before you run command :code:`pydolphinscheduler config --init`.
.. code-block:: bash .. code-block:: bash
$ export PYDS_HOME=<CUSTOM_PATH> export PYDS_HOME=<CUSTOM_PATH>
$ pydolphinscheduler config --init pydolphinscheduler config --init
After that, your configuration file will export into `<CUSTOM_PATH>/config.yaml` instead of the default path. After that, your configuration file will export into `<CUSTOM_PATH>/config.yaml` instead of the default path.
@ -161,36 +161,37 @@ node of the configuration but could get the parent configuration, there are simp
.. code-block:: bash .. code-block:: bash
# Get single configuration in the leaf node # Get single configuration in the leaf node,
$ pydolphinscheduler config --get java_gateway.address # The output look like below:
The configuration query as below: # java_gateway.address = 127.0.0.1
pydolphinscheduler config --get java_gateway.address
java_gateway.address = 127.0.0.1
# Get multiple configuration in the leaf node,
# Get multiple configuration in the leaf node # The output look like below:
$ pydolphinscheduler config --get java_gateway.address --get java_gateway.port # java_gateway.address = 127.0.0.1
The configuration query as below: # java_gateway.port = 25333
pydolphinscheduler config --get java_gateway.address --get java_gateway.port
java_gateway.address = 127.0.0.1
java_gateway.port = 25333
# Get parent configuration which contain multiple leaf nodes,
# Get parent configuration which contain multiple leaf nodes # The output look like below:
$ pydolphinscheduler config --get java_gateway # java_gateway = ordereddict([('address', '127.0.0.1'), ('port', 25333), ('auto_convert', True)])
The configuration query as below: pydolphinscheduler config --get java_gateway
java_gateway = ordereddict([('address', '127.0.0.1'), ('port', 25333), ('auto_convert', True)]) # Set single configuration,
# The output look like below:
# Set single configuration # Set configuration done.
$ pydolphinscheduler config --set java_gateway.address 192.168.1.1 pydolphinscheduler config --set java_gateway.address 192.168.1.1
Set configuration done.
# Set multiple configuration
# Set multiple configuration # The output look like below:
$ pydolphinscheduler config --set java_gateway.address 192.168.1.1 --set java_gateway.port 25334 # Set configuration done.
Set configuration done. pydolphinscheduler config --set java_gateway.address 192.168.1.1 --set java_gateway.port 25334
# Set configuration not in leaf node will fail # Set configuration not in leaf node will fail
$ pydolphinscheduler config --set java_gateway 192.168.1.1,25334,True # The output look like below:
Raise error. # Raise error.
pydolphinscheduler config --set java_gateway 192.168.1.1,25334,True
For more information about our CLI, you could see document :doc:`cli`. For more information about our CLI, you could see document :doc:`cli`.

6
dolphinscheduler-python/pydolphinscheduler/docs/source/howto/remote-submit.rst

@ -28,7 +28,7 @@ Export Configuration File
.. code-block:: bash .. code-block:: bash
$ pydolphinscheduler config --init pydolphinscheduler config --init
your could find more detail in :ref:`configuration exporting <config:export configuration file>` your could find more detail in :ref:`configuration exporting <config:export configuration file>`
@ -37,7 +37,7 @@ Run API Server in Other Host
.. code-block:: bash .. code-block:: bash
$ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname> pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname>
your could find more detail in :ref:`configuration setting <config:change configuration>` your could find more detail in :ref:`configuration setting <config:change configuration>`
@ -46,6 +46,6 @@ Run API Server in Other Port
.. code-block:: bash .. code-block:: bash
$ pydolphinscheduler config --set java_gateway.port <your-python-gateway-service-port> pydolphinscheduler config --set java_gateway.port <your-python-gateway-service-port>
your could find more detail in :ref:`configuration setting <config:change configuration>` your could find more detail in :ref:`configuration setting <config:change configuration>`

49
dolphinscheduler-python/pydolphinscheduler/docs/source/start.rst

@ -41,8 +41,9 @@ without error(here is a example after Python 3.8.7 installed)
.. code-block:: bash .. code-block:: bash
$ python --version python --version
Python 3.8.7
Will see detail of Python version, such as *Python 3.8.7*
Installing PyDolphinScheduler Installing PyDolphinScheduler
----------------------------- -----------------------------
@ -52,7 +53,7 @@ After Python is already installed on your machine following section
.. code-block:: bash .. code-block:: bash
$ pip install apache-dolphinscheduler python -m pip install apache-dolphinscheduler
The latest version of *PyDolphinScheduler* would be installed after you run above The latest version of *PyDolphinScheduler* would be installed after you run above
command in your terminal. You could go and `start Python Gateway Service`_ to finish command in your terminal. You could go and `start Python Gateway Service`_ to finish
@ -60,8 +61,16 @@ the prepare, and then go to :doc:`tutorial` to make your hand dirty. But if you
want to install the unreleased version of *PyDolphinScheduler*, you could go and see want to install the unreleased version of *PyDolphinScheduler*, you could go and see
section `installing PyDolphinScheduler in dev`_ for more detail. section `installing PyDolphinScheduler in dev`_ for more detail.
Installing PyDolphinScheduler In Dev .. note::
------------------------------------
Currently, we released multiple pre-release package in PyPI, you can see all released package
including pre-release in `release history <https://pypi.org/project/apache-dolphinscheduler/#history>`_.
You can fix the the package version if you want to install pre-release package, for example if
you want to install version `3.0.0-beta-2` package, you can run command
:code:`python -m pip install apache-dolphinscheduler==3.0.0b2`.
Installing PyDolphinScheduler In DEV Branch
-------------------------------------------
Because the project is developing and some of the features still not release. Because the project is developing and some of the features still not release.
If you want to try some thing unreleased you could install from the source code If you want to try some thing unreleased you could install from the source code
@ -69,10 +78,10 @@ which we hold in GitHub
.. code-block:: bash .. code-block:: bash
# Clone Apache DolphinScheduler repository # Clone Apache DolphinScheduler repository
$ git clone git@github.com:apache/dolphinscheduler.git git clone git@github.com:apache/dolphinscheduler.git
# Install PyDolphinScheduler in develop mode # Install PyDolphinScheduler in develop mode
$ cd dolphinscheduler-python/pydolphinscheduler && pip install -e . cd dolphinscheduler-python/pydolphinscheduler && python -m pip install -e .
After you installed *PyDolphinScheduler*, please remember `start Python Gateway Service`_ After you installed *PyDolphinScheduler*, please remember `start Python Gateway Service`_
which waiting for *PyDolphinScheduler*'s workflow definition require. which waiting for *PyDolphinScheduler*'s workflow definition require.
@ -88,18 +97,18 @@ go `install Apache DolphinScheduler`_ for more detail
.. code-block:: bash .. code-block:: bash
# Start DolphinScheduler api-server which including python gateway service # Start DolphinScheduler api-server which including python gateway service
$ ./bin/dolphinscheduler-daemon.sh start api-server ./bin/dolphinscheduler-daemon.sh start api-server
To check whether the server is alive or not, you could run :code:`jps`. And To check whether the server is alive or not, you could run :code:`jps`. And
the server is health if keyword `ApiApplicationServer` in the console. the server is health if keyword `ApiApplicationServer` in the console.
.. code-block:: bash .. code-block:: bash
$ jps jps
.... # ....
201472 ApiApplicationServer # 201472 ApiApplicationServer
.... # ....
.. note:: .. note::
@ -115,23 +124,23 @@ single bash command to get it
.. code-block:: bash .. code-block:: bash
$ wget https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py wget https://raw.githubusercontent.com/apache/dolphinscheduler/dev/dolphinscheduler-python/pydolphinscheduler/src/pydolphinscheduler/examples/tutorial.py
or you could copy-paste the content from `tutorial source code`_. And then you could run the example in your or you could copy-paste the content from `tutorial source code`_. And then you could run the example in your
terminal terminal
.. code-block:: bash .. code-block:: bash
$ python tutorial.py python tutorial.py
If you want to submit your workflow to a remote API server, which means that your workflow script is different If you want to submit your workflow to a remote API server, which means that your workflow script is different
from the API server, you should first change pydolphinscheduler configuration and then submit the workflow script from the API server, you should first change pydolphinscheduler configuration and then submit the workflow script
.. code-block:: bash .. code-block:: bash
$ pydolphinscheduler config --init pydolphinscheduler config --init
$ pydolphinscheduler config --set java_gateway.address <your-api-server-ip-or-hostname> pydolphinscheduler config --set java_gateway.address <YOUR-API-SERVER-IP-OR-HOSTNAME>
$ python tutorial.py python tutorial.py
.. note:: .. note::

1
dolphinscheduler-python/pydolphinscheduler/setup.py

@ -52,6 +52,7 @@ doc = [
"sphinx_rtd_theme>=1.0", "sphinx_rtd_theme>=1.0",
"sphinx-click>=3.0", "sphinx-click>=3.0",
"sphinx-inline-tabs", "sphinx-inline-tabs",
"sphinx-copybutton>=0.4.0",
] ]
test = [ test = [

Loading…
Cancel
Save