github
/
nocodb
mirror of https://github.com/nocodb/nocodb
1
0
Fork 0
Code Issues Releases Wiki Activity
Browse Source

Merge pull request #8555 from nocodb/fix/8549-add-support-to-deprecated-envs 

fix: Litestream S3 - add support to deprecated env variables
pull/8566/head
Pranav C 6 months ago committed by GitHub
parent
317f8159e9 f441403aba
commit
bc484166b5
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
4 changed files with 80 additions and 70 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 137
      packages/noco-docs/docs/020.getting-started/050.self-hosted/020.environment-variables.md
  2. 1
      packages/nocodb/Dockerfile
  3. 1
      packages/nocodb/docker/litestream.yml
  4. 11
      packages/nocodb/docker/start-litestream.sh

137
packages/noco-docs/docs/020.getting-started/050.self-hosted/020.environment-variables.md
Unescape View File

@ -13,72 +13,73 @@ For production use cases, it is **recommended** to set at least:
- `NC_PUBLIC_URL`
- `NC_REDIS_URL`
| Variable | Description | If absent |
| -------- | ----------- | --------- |
| `NC_DB` | See our example database URLs [here](https://github.com/nocodb/nocodb#docker). | A local SQLite database is created in root folder if `NC_DB` is not set. |
| `NC_DB_JSON` | Can be used instead of `NC_DB` and value should be valid knex connection JSON string. | |
| `NC_DB_JSON_FILE` | Can be used instead of `NC_DB` and value should be a valid path to knex connection JSON file. | |
| `DATABASE_URL` | Can be used instead of `NC_DB` and value should be a JDBC URL string. | |
| `DATABASE_URL_FILE` | Can be used instead of `NC_DB` and value should be a valid path to a JDBC URL file. | |
| `NC_AUTH_JWT_SECRET` | JWT secret used for auth and storing other secrets. | A random secret is generated. |
| `NC_ADMIN_EMAIL` | Super admin e-mail address. | |
| `NC_ADMIN_PASSWORD` | Super admin password. The password should have at least 8 letters with one uppercase, one number and one special letter. Allowed special characters include `$&+,:;=?@#\|'.^*()%!_-"`. | |
| `PORT` | Network port NocoDB runs on. | Defaults to `8080`. |
| `DB_QUERY_LIMIT_DEFAULT` | Pagination limit. | Defaults to `25`. |
| `DB_QUERY_LIMIT_GROUP_BY_GROUP` | Group per page limit. | Defaults to `10`. |
| `DB_QUERY_LIMIT_GROUP_BY_RECORD` | Record per group limit. | Defaults to `10`. |
| `DB_QUERY_LIMIT_MAX` | Maximum allowed pagination limit. | Defaults to `1000`. |
| `DB_QUERY_LIMIT_MIN` | Minimum allowed pagination limit. | Defaults to `1`. |
| Variable | Description | If absent |
| -------- | ----------- |-----------------------------------------------------------------------------------------------------|
| `NC_DB` | See our example database URLs [here](https://github.com/nocodb/nocodb#docker). | A local SQLite database is created in root folder if `NC_DB` is not set. |
| `NC_DB_JSON` | Can be used instead of `NC_DB` and value should be valid knex connection JSON string. | |
| `NC_DB_JSON_FILE` | Can be used instead of `NC_DB` and value should be a valid path to knex connection JSON file. | |
| `DATABASE_URL` | Can be used instead of `NC_DB` and value should be a JDBC URL string. | |
| `DATABASE_URL_FILE` | Can be used instead of `NC_DB` and value should be a valid path to a JDBC URL file. | |
| `NC_AUTH_JWT_SECRET` | JWT secret used for auth and storing other secrets. | A random secret is generated. |
| `NC_ADMIN_EMAIL` | Super admin e-mail address. | |
| `NC_ADMIN_PASSWORD` | Super admin password. The password should have at least 8 letters with one uppercase, one number and one special letter. Allowed special characters include `$&+,:;=?@#\|'.^*()%!_-"`. | |
| `PORT` | Network port NocoDB runs on. | Defaults to `8080`. |
| `DB_QUERY_LIMIT_DEFAULT` | Pagination limit. | Defaults to `25`. |
| `DB_QUERY_LIMIT_GROUP_BY_GROUP` | Group per page limit. | Defaults to `10`. |
| `DB_QUERY_LIMIT_GROUP_BY_RECORD` | Record per group limit. | Defaults to `10`. |
| `DB_QUERY_LIMIT_MAX` | Maximum allowed pagination limit. | Defaults to `1000`. |
| `DB_QUERY_LIMIT_MIN` | Minimum allowed pagination limit. | Defaults to `1`. |
| `NC_TOOL_DIR` | App directory to keep metadata and app related files in. | Defaults to the current working directory. In docker, maps to `/usr/app/data/` for mounting volume. |
| `NC_PUBLIC_URL` | Used for sending E-mail invitations. | Best guess from HTTP request params. |
| `NC_JWT_EXPIRES_IN` | JWT token expiry time | Defaults to `10h`. |
| `NC_CONNECT_TO_EXTERNAL_DB_DISABLED` | Disable base creation on external databases. | |
| `NC_MINIMAL_DBS` | Create a new SQLite file for each base. All the SQLite database files are stored in the `nc_minimal_dbs` folder in the current working directory. Enabling this option automatically sets `NC_CONNECT_TO_EXTERNAL_DB_DISABLED`, i.e. disables base creation on external databases. | |
| `NC_INVITE_ONLY_SIGNUP` | Removed since version 0.99.0, and now it's recommended to use the [super admin settings menu](/account-settings/oss-specific-details#enable--disable-signup). Disable public signup and allow signup only via invitations. | |
| `NUXT_PUBLIC_NC_BACKEND_URL` | Custom backend URL. | Defaults to `http://localhost:8080`. |
| `NC_REQUEST_BODY_SIZE` | Request body size [limit](https://expressjs.com/en/resources/middleware/body-parser.html#limit) | Defaults to `1048576`. |
| `NC_EXPORT_MAX_TIMEOUT` | After `NC_EXPORT_MAX_TIMEOUT` (in milliseconds), CSV gets downloaded in batches. | Defaults to `5000` (5 seconds). |
| `NC_DISABLE_TELE` | Disable telemetry. | |
| `NC_DASHBOARD_URL` | Custom dashboard URL path | Defaults to `/dashboard`. |
| `NC_GOOGLE_CLIENT_ID` | Google client ID to enable Google authentication. | |
| `NC_GOOGLE_CLIENT_SECRET` | Google client secret to enable Google authentication. | |
| `NC_MIGRATIONS_DISABLED` | Disable NocoDB migrations. | |
| `NC_MIN` | Disable default splash screen (initial welcome animation) and matrix screensaver. | |
| `NC_SENTRY_DSN` | Data Source Name (DSN) for Sentry monitoring. | |
| `NC_REDIS_URL` | Redis URL. Example: `redis://:authpassword@127.0.0.1:6380/4` | Meta data is stored in memory. |
| `NC_DISABLE_ERR_REPORT` | Disable error reporting. | |
| `NC_DISABLE_CACHE` | Disable cache. To be used only while debugging. If `true`, meta data is fetched from database instead of redis/cache. | Defaults to `false`. |
| `NC_SMTP_FROM` | E-mail sender address for SMTP plugin. | *SMTP plugin is disabled if this variable is not set.* |
| `NC_SMTP_HOST` | E-mail server hostname for SMTP plugin. | *SMTP plugin is disabled if this variable is not set.* |
| `NC_SMTP_PORT` | E-mail server network for SMTP plugin. | *SMTP plugin is disabled if this variable is not set.* |
| `NC_SMTP_USERNAME` | Username for authentication in SMTP plugin. | |
| `NC_SMTP_PASSWORD` | Password for authentication in SMTP plugin. | |
| `NC_SMTP_SECURE` | Enable secure authentication in SMTP plugin. Set to `true` to enable, any other value is treated as `false`. | |
| `NC_SMTP_IGNORE_TLS` | Ignore TLS in SMTP plugin. Set to `true` to ignore TLS, any other value is treated as `false`. For more information, visit [Nodemailer's SMTP documentation](https://nodemailer.com/smtp/). | |
| `NC_S3_BUCKET_NAME` | AWS S3 bucket name for S3 storage plugin. | |
| `NC_S3_REGION` | AWS S3 region for S3 storage plugin. | |
| `NC_S3_ACCESS_KEY` | AWS access key ID for S3 storage plugin. | |
| `NC_S3_ACCESS_SECRET` | AWS access secret for S3 storage plugin. | |
| `NC_ATTACHMENT_FIELD_SIZE` | Maximum file size for [attachments](/fields/field-types/custom-types/attachment/) in bytes. | Defaults to `20971520` (20 MiB). |
| `NC_MAX_ATTACHMENTS_ALLOWED` | Maximum number of attachments per cell. | Defaults to `10`. |
| `NC_SECURE_ATTACHMENTS` | Allow accessing attachments only through pre-signed URLs. Set to `true` to enable, any other value is treated as `false`. (⚠ this will make existing links inaccessible ⚠) | Defaults to `false`. |
| `NC_ATTACHMENT_EXPIRE_SECONDS` | Number of seconds after which pre-signed attachment URLs will begin to expire. The URLs will expire after `NC_ATTACHMENT_EXPIRE_SECONDS` plus 10 minutes at the very latest. | Defaults to `7200` (2 hours). |
| `NC_DISABLE_AUDIT` | Disable audit log. | Defaults to `false`. |
| `NC_AUTOMATION_LOG_LEVEL` | Possible Values: `OFF`, `ERROR`, `ALL`. See [Webhooks](/automation/webhook/create-webhook#call-log) for details. | Defaults to `OFF`. |
| `NC_ALLOW_LOCAL_HOOKS` | ⚠ Allow webhooks to call local links, which can raise security issues. ⚠ Set to `true` to enable, any other value is treated as `false` | Defaults to `false`. |
| `NC_SANITIZE_COLUMN_NAME` | Sanitize the column name during column creation. Set to `true` to enable, any other value is treated as `false` | Defaults to `true`. |
| `NODE_OPTIONS` | Node.js [options](https://nodejs.org/api/cli.html#node_optionsoptions) to pass to instance. | |
| `LITESTREAM_S3_ENDPOINT` | URL of an S3-compatible object storage service endpoint for [Litestream](https://litestream.io/) replication of NocoDB's default SQLite database. Example: `s3.eu-central-1.amazonaws.com` | *Litestream replication is disabled if this variable is not set.* |
| `LITESTREAM_S3_BUCKET` | Name of the object storage bucket to store the Litestream replication in. | *Litestream replication is disabled if this variable is not set.* |
| `LITESTREAM_S3_PATH` | Directory path to use within the Litestream replication object storage bucket. | Defaults to `nocodb`. |
| `LITESTREAM_S3_ACCESS_KEY_ID` | Authentication key ID for the Litestream replication object storage bucket. | *Litestream replication is disabled if this variable is not set.* |
| `LITESTREAM_S3_SECRET_ACCESS_KEY` | Authentication secret for the Litestream replication object storage bucket. | *Litestream replication is disabled if this variable is not set.* |
| `LITESTREAM_S3_SKIP_VERIFY` | Whether to disable TLS verification for the Litestream replication object storage service. This is useful when testing against a local node such as MinIO and you are using self-signed certificates. | Defaults to `false`. |
| `LITESTREAM_RETENTION` | Amount of time Litestream snapshot and WAL files are kept. After the retention period, a new snapshot is created and the old one is removed. WAL files that exist before the oldest snapshot will also be removed. | Defaults to `1440h` (60 days). |
| `LITESTREAM_RETENTION_CHECK_INTERVAL` | Frequency in which Litestream will check if retention needs to be enforced. | Defaults to `72h` (3 days). |
| `LITESTREAM_SNAPSHOT_INTERVAL` | Frequency in which new Litestream snapshots are created. A higher frequency reduces the time to restore since newer snapshots will have fewer WAL frames to apply. Retention still applies to these snapshots. | Defaults to `24h` (1 day). |
| `LITESTREAM_SYNC_INTERVAL` | Frequency in which frames are pushed to the Litestream replica. Increasing this frequency can increase object storage costs significantly. | Defaults to `60s` (1 minute). |
| `LITESTREAM_AGE_PUBLIC_KEY` | [age](https://age-encryption.org/) public key generated by `age-keygen` (`age1...`) or SSH public key (`ssh-ed25519 AAAA...`, `ssh-rsa AAAA...`) used to encrypt the Litestream replication for. Refer to the relevant [Litestream documentation](https://litestream.io/reference/config/#encryption) for details. | *Litestream replication is unencrypted if this variable is not set.* |
| `LITESTREAM_AGE_SECRET_KEY` | [age](https://age-encryption.org/) secret key (`AGE-SECRET-KEY-1...`) used to encrypt the Litestream replication with. Refer to the relevant [Litestream documentation](https://litestream.io/reference/config/#encryption) for details. | *Litestream replication is unencrypted if this variable is not set.* |
| `AWS_ACCESS_KEY_ID` | ***Deprecated***. Please use `LITESTREAM_S3_ACCESS_KEY_ID` instead. | |
| `AWS_SECRET_ACCESS_KEY` | ***Deprecated***. Please use `LITESTREAM_S3_SECRET_ACCESS_KEY` instead. | |
| `NC_PUBLIC_URL` | Used for sending E-mail invitations. | Best guess from HTTP request params. |
| `NC_JWT_EXPIRES_IN` | JWT token expiry time | Defaults to `10h`. |
| `NC_CONNECT_TO_EXTERNAL_DB_DISABLED` | Disable base creation on external databases. | |
| `NC_MINIMAL_DBS` | Create a new SQLite file for each base. All the SQLite database files are stored in the `nc_minimal_dbs` folder in the current working directory. Enabling this option automatically sets `NC_CONNECT_TO_EXTERNAL_DB_DISABLED`, i.e. disables base creation on external databases. | |
| `NC_INVITE_ONLY_SIGNUP` | Removed since version 0.99.0, and now it's recommended to use the [super admin settings menu](/account-settings/oss-specific-details#enable--disable-signup). Disable public signup and allow signup only via invitations. | |
| `NUXT_PUBLIC_NC_BACKEND_URL` | Custom backend URL. | Defaults to `http://localhost:8080`. |
| `NC_REQUEST_BODY_SIZE` | Request body size [limit](https://expressjs.com/en/resources/middleware/body-parser.html#limit) | Defaults to `1048576`. |
| `NC_EXPORT_MAX_TIMEOUT` | After `NC_EXPORT_MAX_TIMEOUT` (in milliseconds), CSV gets downloaded in batches. | Defaults to `5000` (5 seconds). |
| `NC_DISABLE_TELE` | Disable telemetry. | |
| `NC_DASHBOARD_URL` | Custom dashboard URL path | Defaults to `/dashboard`. |
| `NC_GOOGLE_CLIENT_ID` | Google client ID to enable Google authentication. | |
| `NC_GOOGLE_CLIENT_SECRET` | Google client secret to enable Google authentication. | |
| `NC_MIGRATIONS_DISABLED` | Disable NocoDB migrations. | |
| `NC_MIN` | Disable default splash screen (initial welcome animation) and matrix screensaver. | |
| `NC_SENTRY_DSN` | Data Source Name (DSN) for Sentry monitoring. | |
| `NC_REDIS_URL` | Redis URL. Example: `redis://:authpassword@127.0.0.1:6380/4` | Meta data is stored in memory. |
| `NC_DISABLE_ERR_REPORT` | Disable error reporting. | |
| `NC_DISABLE_CACHE` | Disable cache. To be used only while debugging. If `true`, meta data is fetched from database instead of redis/cache. | Defaults to `false`. |
| `NC_SMTP_FROM` | E-mail sender address for SMTP plugin. | *SMTP plugin is disabled if this variable is not set.* |
| `NC_SMTP_HOST` | E-mail server hostname for SMTP plugin. | *SMTP plugin is disabled if this variable is not set.* |
| `NC_SMTP_PORT` | E-mail server network for SMTP plugin. | *SMTP plugin is disabled if this variable is not set.* |
| `NC_SMTP_USERNAME` | Username for authentication in SMTP plugin. | |
| `NC_SMTP_PASSWORD` | Password for authentication in SMTP plugin. | |
| `NC_SMTP_SECURE` | Enable secure authentication in SMTP plugin. Set to `true` to enable, any other value is treated as `false`. | |
| `NC_SMTP_IGNORE_TLS` | Ignore TLS in SMTP plugin. Set to `true` to ignore TLS, any other value is treated as `false`. For more information, visit [Nodemailer's SMTP documentation](https://nodemailer.com/smtp/). | |
| `NC_S3_BUCKET_NAME` | AWS S3 bucket name for S3 storage plugin. | |
| `NC_S3_REGION` | AWS S3 region for S3 storage plugin. | |
| `NC_S3_ACCESS_KEY` | AWS access key ID for S3 storage plugin. | |
| `NC_S3_ACCESS_SECRET` | AWS access secret for S3 storage plugin. | |
| `NC_ATTACHMENT_FIELD_SIZE` | Maximum file size for [attachments](/fields/field-types/custom-types/attachment/) in bytes. | Defaults to `20971520` (20 MiB). |
| `NC_MAX_ATTACHMENTS_ALLOWED` | Maximum number of attachments per cell. | Defaults to `10`. |
| `NC_SECURE_ATTACHMENTS` | Allow accessing attachments only through pre-signed URLs. Set to `true` to enable, any other value is treated as `false`. (⚠ this will make existing links inaccessible ⚠) | Defaults to `false`. |
| `NC_ATTACHMENT_EXPIRE_SECONDS` | Number of seconds after which pre-signed attachment URLs will begin to expire. The URLs will expire after `NC_ATTACHMENT_EXPIRE_SECONDS` plus 10 minutes at the very latest. | Defaults to `7200` (2 hours). |
| `NC_DISABLE_AUDIT` | Disable audit log. | Defaults to `false`. |
| `NC_AUTOMATION_LOG_LEVEL` | Possible Values: `OFF`, `ERROR`, `ALL`. See [Webhooks](/automation/webhook/create-webhook#call-log) for details. | Defaults to `OFF`. |
| `NC_ALLOW_LOCAL_HOOKS` | ⚠ Allow webhooks to call local links, which can raise security issues. ⚠ Set to `true` to enable, any other value is treated as `false` | Defaults to `false`. |
| `NC_SANITIZE_COLUMN_NAME` | Sanitize the column name during column creation. Set to `true` to enable, any other value is treated as `false` | Defaults to `true`. |
| `NODE_OPTIONS` | Node.js [options](https://nodejs.org/api/cli.html#node_optionsoptions) to pass to instance. | |
| `LITESTREAM_S3_ENDPOINT` | URL of an S3-compatible object storage service endpoint for [Litestream](https://litestream.io/) replication of NocoDB's default SQLite database. Example: `s3.eu-central-1.amazonaws.com` | Fallback to default AWS s3 endpoint |
| `LITESTREAM_S3_REGION` | AWS region of the Litestream replication object storage bucket. | If not configured it will fallback to the default region, check litestream docs for more info |
| `LITESTREAM_S3_BUCKET` | Name of the object storage bucket to store the Litestream replication in. | *Litestream replication is disabled if this variable is not set.* |
| `LITESTREAM_S3_PATH` | Directory path to use within the Litestream replication object storage bucket. | Defaults to `nocodb`. |
| `LITESTREAM_S3_ACCESS_KEY_ID` | Authentication key ID for the Litestream replication object storage bucket. | *Litestream replication is disabled if this variable is not set.* |
| `LITESTREAM_S3_SECRET_ACCESS_KEY` | Authentication secret for the Litestream replication object storage bucket. | *Litestream replication is disabled if this variable is not set.* |
| `LITESTREAM_S3_SKIP_VERIFY` | Whether to disable TLS verification for the Litestream replication object storage service. This is useful when testing against a local node such as MinIO and you are using self-signed certificates. | Defaults to `false`. |
| `LITESTREAM_RETENTION` | Amount of time Litestream snapshot and WAL files are kept. After the retention period, a new snapshot is created and the old one is removed. WAL files that exist before the oldest snapshot will also be removed. | Defaults to `1440h` (60 days). |
| `LITESTREAM_RETENTION_CHECK_INTERVAL` | Frequency in which Litestream will check if retention needs to be enforced. | Defaults to `72h` (3 days). |
| `LITESTREAM_SNAPSHOT_INTERVAL` | Frequency in which new Litestream snapshots are created. A higher frequency reduces the time to restore since newer snapshots will have fewer WAL frames to apply. Retention still applies to these snapshots. | Defaults to `24h` (1 day). |
| `LITESTREAM_SYNC_INTERVAL` | Frequency in which frames are pushed to the Litestream replica. Increasing this frequency can increase object storage costs significantly. | Defaults to `60s` (1 minute). |
| `LITESTREAM_AGE_PUBLIC_KEY` | [age](https://age-encryption.org/) public key generated by `age-keygen` (`age1...`) or SSH public key (`ssh-ed25519 AAAA...`, `ssh-rsa AAAA...`) used to encrypt the Litestream replication for. Refer to the relevant [Litestream documentation](https://litestream.io/reference/config/#encryption) for details. | *Litestream replication is unencrypted if this variable is not set.* |
| `LITESTREAM_AGE_SECRET_KEY` | [age](https://age-encryption.org/) secret key (`AGE-SECRET-KEY-1...`) used to encrypt the Litestream replication with. Refer to the relevant [Litestream documentation](https://litestream.io/reference/config/#encryption) for details. | *Litestream replication is unencrypted if this variable is not set.* |
| `AWS_ACCESS_KEY_ID` | ***Deprecated***. Please use `LITESTREAM_S3_ACCESS_KEY_ID` instead. | |
| `AWS_SECRET_ACCESS_KEY` | ***Deprecated***. Please use `LITESTREAM_S3_SECRET_ACCESS_KEY` instead. | |

1
packages/nocodb/Dockerfile
Unescape View File

@ -51,7 +51,6 @@ FROM alpine:3.19
WORKDIR /usr/src/app
ENV LITESTREAM_S3_SKIP_VERIFY=false \
LITESTREAM_S3_PATH=nocodb \
LITESTREAM_RETENTION=1440h \
LITESTREAM_RETENTION_CHECK_INTERVAL=72h \
LITESTREAM_SNAPSHOT_INTERVAL=24h \

1
packages/nocodb/docker/litestream.yml
Unescape View File

@ -6,6 +6,7 @@ dbs:
- type: s3
endpoint: ${LITESTREAM_S3_ENDPOINT}
force-path-style: true
region: ${LITESTREAM_S3_REGION}
skip-verify: ${LITESTREAM_S3_SKIP_VERIFY}
bucket: ${LITESTREAM_S3_BUCKET}
path: ${LITESTREAM_S3_PATH}

11
packages/nocodb/docker/start-litestream.sh
Unescape View File

@ -11,6 +11,13 @@ fi
if [ -z "${LITESTREAM_S3_SECRET_ACCESS_KEY}" ] && [ -n "${AWS_SECRET_ACCESS_KEY}" ] ; then
export LITESTREAM_S3_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}"
fi
if [ -z "${LITESTREAM_S3_PATH}" ] && -n [ "${AWS_BUCKET_PATH}" ] ; then
export LITESTREAM_S3_PATH="${AWS_BUCKET_PATH}"
fi
if [ -z "${LITESTREAM_S3_BUCKET}" ] && [ -n "${AWS_BUCKET}" ] ; then
export LITESTREAM_S3_BUCKET="${AWS_BUCKET}"
fi
use_litestream() {
[ -z "${NC_DB}" ] \
@ -19,7 +26,6 @@ use_litestream() {
&& [ -z "${DATABASE_URL}" ] \
&& [ -z "${DATABASE_URL_FILE}" ] \
&& [ -z "${NC_MINIMAL_DBS}" ] \
&& [ -n "${LITESTREAM_S3_ENDPOINT}" ] \
&& [ -n "${LITESTREAM_S3_BUCKET}" ] \
&& [ -n "${LITESTREAM_S3_ACCESS_KEY_ID}" ] \
&& [ -n "${LITESTREAM_S3_SECRET_ACCESS_KEY}" ]
@ -27,6 +33,9 @@ use_litestream() {
if use_litestream ; then
# set default bucket path if not provided
: "${LITESTREAM_S3_PATH:=nocodb}"
# enable age encryption in Litestream config if indicated
LITESTREAM_CONFIG_PATH='/etc/litestream.yml'

Write Preview
Loading…
Cancel
Save