github
/
nocodb
mirror of https://github.com/nocodb/nocodb
1
0
Fork 0
Code Issues Releases Wiki Activity
Browse Source

Merge pull request #9642 from Upgreydd/issue-6819-aws-access-iam 

AWS IAM Role support
pull/9659/head
navi 1 month ago committed by GitHub
parent
93312073d9 ae7b58c1e7
commit
0bae6bb6fd
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
5 changed files with 148 additions and 132 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 56
      packages/noco-docs/docs/020.getting-started/050.self-hosted/020.environment-variables.md
  2. 21
      packages/nocodb/src/helpers/NcPluginMgrv2.ts
  3. 12
      packages/nocodb/src/plugins/GenericS3/GenericS3.ts
  4. 15
      packages/nocodb/src/plugins/s3/S3.ts
  5. 4
      packages/nocodb/src/plugins/s3/index.ts

56
packages/noco-docs/docs/020.getting-started/050.self-hosted/020.environment-variables.md
Unescape View File

@ -1,15 +1,17 @@
---
title: 'Environment variables'
description: 'Environment Variables for NocoDB!'
tags: ['Open Source']
keywords : ['NocoDB Environment Variables', 'NocoDB env variables', 'NocoDB envs', 'NocoDB .env']
tags: [ 'Open Source' ]
keywords: [ 'NocoDB Environment Variables', 'NocoDB env variables', 'NocoDB envs', 'NocoDB .env' ]
---
For production use cases, it is crucial to set all environment variables marked as **"Mandatory"** to ensure optimal performance, security, and functionality of NocoDB.
For production use cases, it is crucial to set all environment variables marked as **"Mandatory"** to ensure optimal
performance, security, and functionality of NocoDB.
## Database
| Variable | Mandatory | Description | If Not Set |
| -------- |-----------| ----------- | ---------- |
|-----------------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
| `NC_DB` | Yes | The primary database where all NocoDB metadata and data are stored. Example format: `pg://host.docker.internal:5432?u=username&p=password&d=database_name`. | A local SQLite database will be created in the root folder if `NC_DB` is not specified. |
| `NC_DB_JSON` | No | Allows setting the database connection using a valid [knex connection JSON string](https://knexjs.org/guide/#configuration-options) instead of `NC_DB`. | |
| `NC_DB_JSON_FILE` | No | A path to a knex connection JSON file can be used to specify the database connection, as an alternative to `NC_DB`. | |
@ -18,8 +20,9 @@ For production use cases, it is crucial to set all environment variables marked
| `NC_CONNECTION_ENCRYPT_KEY` | No | The key used to encrypt the credentials of external databases. <br/> **Warning:** Changing this variable may break the application. If you must change it, use the CLI as described in the [NocoDB Secret CLI documentation](/data-sources/updating-secret). | Keep connection credentials as plain text in the database if not set. |
## Authentication
| Variable | Mandatory | Description | If Not Set |
| -------- |-----------| ----------- | ---------- |
|---------------------------|-----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------|
| `NC_AUTH_JWT_SECRET` | Yes | This JWT secret is utilized for generating authentication tokens. | A random secret will be generated automatically. |
| `NC_JWT_EXPIRES_IN` | No | Specifies the expiration time for JWT tokens. | Defaults to `10h`. |
| `NC_GOOGLE_CLIENT_ID` | No | Google client ID required to activate Google authentication. | |
@ -29,15 +32,16 @@ For production use cases, it is crucial to set all environment variables marked
| `NC_DISABLE_EMAIL_AUTH` | No | Disables email and password-based authentication, intended for use when Google authentication variables are configured. | |
## Storage
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|
|--------------------------------|-----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|
| `NC_S3_BUCKET_NAME` | No | The name of the AWS S3 bucket used for the S3 storage plugin. | |
| `NC_S3_REGION` | No | The AWS S3 region where the S3 storage plugin bucket is located. | |
| `NC_S3_ENDPOINT` | No | S3 endpoint for S3 storage plugin. | Defaults to `s3.<region>.amazonaws.com` |
| `NC_S3_ACCESS_KEY` | No | The AWS access key ID required for the S3 storage plugin. | |
| `NC_S3_ACCESS_SECRET` | No | The AWS access secret associated with the S3 storage plugin. | |
| `NC_S3_ENDPOINT` | No | S3 endpoint for S3 storage plugin. | Defaults to `s3.<region>.amazonaws.com` |
| `NC_S3_ACCESS_KEY` | No | The AWS access key ID for the S3 storage plugin. Required if no role access in use. | |
| `NC_S3_ACCESS_SECRET` | No | The AWS access secret associated with the S3 storage plugin. Required if no role access in use. | |
| `NC_S3_FORCE_PATH_STYLE` | No | Whether to force [path-style requests](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access) for the S3 storage plugin. | |
| `NC_S3_ACL` | No | The [ACL](https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html) for the objects in S3 | | |
| `NC_S3_ACL` | No | The [ACL](https://docs.aws.amazon.com/AmazonS3/latest/userguide/acl-overview.html) for the objects in S3 | |
| `NC_ATTACHMENT_FIELD_SIZE` | No | Maximum file size allowed for [attachments](/fields/field-types/custom-types/attachment/) in bytes. | Defaults to `20971520` (20 MiB). |
| `NC_MAX_ATTACHMENTS_ALLOWED` | No | Maximum number of attachments allowed per cell. | Defaults to `10`. |
| `NC_ATTACHMENT_RETENTION_DAYS` | No | Number of days to retain attachment on storage after all references deleted. (Set 0 to keep forever) | Defaults to `10`. |
@ -45,10 +49,11 @@ For production use cases, it is crucial to set all environment variables marked
| `NC_ATTACHMENT_EXPIRE_SECONDS` | No | Time in seconds after which pre-signed URLs for attachments start to expire. The actual expiration will occur after this time plus an additional 10 minutes. Only applicable if `NC_SECURE_ATTACHMENTS` is enabled. | Defaults to `7200` (2 hours). |
## Email Notifications
- The following SMTP variables are used to send email notifications to users, e.g., invites.
| Variable | Mandatory | Description | If Not Set |
| -------- |-----------| ----------- | ---------- |
|----------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|
| `NC_SMTP_FROM` | Yes | The email address used as the sender for the SMTP plugin. | |
| `NC_SMTP_HOST` | Yes | The hostname of the email server for the SMTP plugin. | |
| `NC_SMTP_PORT` | Yes | The network port of the email server for the SMTP plugin. | |
@ -58,26 +63,30 @@ For production use cases, it is crucial to set all environment variables marked
| `NC_SMTP_IGNORE_TLS` | Yes | Ignores TLS for the SMTP plugin. Set to `true` to ignore TLS; all other values are considered `false`. For more details, see [Nodemailer's SMTP documentation](https://nodemailer.com/smtp/). | |
## Backend
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- | ----------- | ---------- |
|----------------|-----------|-------------------------------------------------------------------------------------------------|---------------------|
| `PORT` | No | Specifies the network port on which NocoDB will run. | Defaults to `8080`. |
| `NODE_OPTIONS` | No | Node.js [options](https://nodejs.org/api/cli.html#node_optionsoptions) to pass to the instance. | |
## Frontend
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- | ----------- | ---------- |
|------------------------------|-----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|
| `NC_PUBLIC_URL` | No | This is the base URL used for constructing URLs in email templates, generating the Swagger documentation URL, and handling backend URL requirements. It should be set to your public-facing NocoDB URL to ensure consistency across the application. | By default, it infers the URL from the incoming request on the backend. If the server is behind a proxy, this may result in incorrect URLs. |
| `NC_DASHBOARD_URL` | No | Defines a custom dashboard URL path. | Defaults to `/dashboard`. |
| `NUXT_PUBLIC_NC_BACKEND_URL` | No | Specifies a custom backend URL. | Defaults to `http://localhost:8080`. |
## Cache
| Variable | Mandatory | Description | If Not Set |
| -------- |-----------|--------------------------------------------------------------------------------------------------|--------------------------|
|----------------|-----------|--------------------------------------------------------------------------------------------------|--------------------------|
| `NC_REDIS_URL` | Yes | Specifies the Redis URL used for caching. <br></br> Eg: `redis://:authpassword@127.0.0.1:6380/4` | Caching layer of backend |
## Product Configuration
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------|
|--------------------------------------|-----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------|
| `DB_QUERY_LIMIT_DEFAULT` | No | Default pagination limit for data tables. | Defaults to `25`. Maximum is `100` |
| `DB_QUERY_LIMIT_GROUP_BY_GROUP` | No | Number of groups per page. | Defaults to `10`. |
| `DB_QUERY_LIMIT_GROUP_BY_RECORD` | No | Number of records per group. | Defaults to `10`. |
@ -97,30 +106,35 @@ For production use cases, it is crucial to set all environment variables marked
### Postgres Data Reflection
NocoDB UI is exactly what's in your Postgres database schema. Same tables, same columns—everything is perfectly mirrored. This is done by creating a schema for each base in PostgreSQL. This feature is enabled by default if the user has the required permissions. To disable it, set the `NC_DISABLE_PG_DATA_REFLECTION` environment variable to `false`.
NocoDB UI is exactly what's in your Postgres database schema. Same tables, same columns—everything is perfectly
mirrored. This is done by creating a schema for each base in PostgreSQL. This feature is enabled by default if the user
has the required permissions. To disable it, set the `NC_DISABLE_PG_DATA_REFLECTION` environment variable to `false`.
## Logging & Monitoring
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- |---------------------------------------------------------------------------------------|------------|
|--------------------------|-----------|---------------------------------------------------------------------------------------|------------|
| `NC_SENTRY_DSN` | No | Data Source Name (DSN) for integrating with Sentry for monitoring and error tracking. | |
| `NC_DISABLE_ERR_REPORTS` | No | Disable default Sentry error reporting. | TRUE |
## Debugging Only
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- | ----------- | ---------- |
|--------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------------|----------------------|
| `NC_DISABLE_CACHE` | No | Disables caching to force metadata fetching directly from the database instead of Redis/cache. Recommended only during debugging. | Defaults to `false`. |
## Telemetry
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- | ----------- | ---------- |
|-------------------|-----------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------|
| `NC_DISABLE_TELE` | No | Disables the telemetry to prevent sending anonymous usage data. Please keep it enabled to help us understand the usage of the product and the impact that any new breaking change can cause. | |
## Litestream
> Litestream is used **only** when `NC_DB` is set to SQLite. It backs up the SQLite database and stores it in S3.
| Variable | Mandatory | Description | If Not Set |
| -------- | --------- | ----------- | ---------- |
|---------------------------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------|
| `LITESTREAM_S3_ENDPOINT` | No | URL of an S3-compatible object storage service endpoint for [Litestream](https://litestream.io/) replication of NocoDB's default SQLite database. Example: `s3.eu-central-1.amazonaws.com`. | Defaults to [AWS S3](https://aws.amazon.com/s3/). |
| `LITESTREAM_S3_REGION` | No | AWS region of the Litestream replication object storage bucket. Note that `LITESTREAM_S3_ENDPOINT` takes precedence if configured (the endpoint URL includes the region). | Defaults to the [default region configured in AWS](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-region.html). |
| `LITESTREAM_S3_BUCKET` | No | Name of the object storage bucket to store the Litestream replication in. | *Litestream replication is disabled if this variable is not set.* |

21
packages/nocodb/src/helpers/NcPluginMgrv2.ts
Unescape View File

@ -118,28 +118,27 @@ class NcPluginMgrv2 {
/*
* NC_S3_BUCKET_NAME
* NC_S3_REGION
* NC_S3_ACCESS_KEY
* NC_S3_ACCESS_SECRET
* */
if (
process.env.NC_S3_BUCKET_NAME &&
process.env.NC_S3_REGION &&
process.env.NC_S3_ACCESS_KEY &&
process.env.NC_S3_ACCESS_SECRET
process.env.NC_S3_REGION
) {
const s3Plugin = await Plugin.getPluginByTitle(S3PluginConfig.title);
await Plugin.update(s3Plugin.id, {
active: true,
input: JSON.stringify({
const s3CfgData: Record<string, any> = {
bucket: process.env.NC_S3_BUCKET_NAME,
region: process.env.NC_S3_REGION,
endpoint: process.env.NC_S3_ENDPOINT,
access_key: process.env.NC_S3_ACCESS_KEY,
access_secret: process.env.NC_S3_ACCESS_SECRET,
force_path_style: process.env.NC_S3_FORCE_PATH_STYLE === 'true',
acl: process.env.NC_S3_ACL,
}),
}
if (process.env.NC_S3_ACCESS_KEY && process.env.NC_S3_ACCESS_SECRET) {
s3CfgData.access_key = process.env.NC_S3_ACCESS_KEY
s3CfgData.access_secret = process.env.NC_S3_ACCESS_SECRET
}
await Plugin.update(s3Plugin.id, {
active: true,
input: JSON.stringify(s3CfgData),
});
}

12
packages/nocodb/src/plugins/GenericS3/GenericS3.ts
Unescape View File

@ -14,21 +14,21 @@ import type { PutObjectRequest, S3 as S3Client } from '@aws-sdk/client-s3';
import type { IStorageAdapterV2, XcFile } from '~/types/nc-plugin';
import { generateTempFilePath, waitForStreamClose } from '~/utils/pluginUtils';
interface GenerocObjectStorageInput {
interface GenericObjectStorageInput {
bucket: string;
region?: string;
access_key: string;
access_secret: string;
access_key?: string;
access_secret?: string;
}
export default class GenericS3 implements IStorageAdapterV2 {
public name;
protected s3Client: S3Client;
protected input: GenerocObjectStorageInput;
protected input: GenericObjectStorageInput;
constructor(input: unknown) {
this.input = input as GenerocObjectStorageInput;
constructor(input: GenericObjectStorageInput) {
this.input = input;
}
protected get defaultParams() {

15
packages/nocodb/src/plugins/s3/S3.ts
Unescape View File

@ -7,8 +7,8 @@ import GenericS3 from '~/plugins/GenericS3/GenericS3';
interface S3Input {
bucket: string;
region: string;
access_key: string;
access_secret: string;
access_key?: string;
access_secret?: string;
endpoint?: string;
acl?: string;
force_path_style?: boolean;
@ -48,13 +48,16 @@ export default class S3 extends GenericS3 implements IStorageAdapterV2 {
public async init(): Promise<any> {
const s3Options: S3ClientConfig = {
region: this.input.region,
credentials: {
accessKeyId: this.input.access_key,
secretAccessKey: this.input.access_secret,
},
forcePathStyle: this.input.force_path_style ?? false,
};
if (this.input.access_key && this.input.access_secret) {
s3Options.credentials = {
accessKeyId: this.input.access_key,
secretAccessKey: this.input.access_secret,
}
}
if (this.input.endpoint) {
s3Options.endpoint = this.input.endpoint;
}

4
packages/nocodb/src/plugins/s3/index.ts
Unescape View File

@ -38,14 +38,14 @@ const config: XcPluginConfig = {
label: 'Access Key',
placeholder: 'Access Key',
type: XcType.SingleLineText,
required: true,
required: false,
},
{
key: 'access_secret',
label: 'Access Secret',
placeholder: 'Access Secret',
type: XcType.Password,
required: true,
required: false,
},
{
key: 'acl',

Write Preview
Loading…
Cancel
Save