File Connectors for Data Sources

This system provides a standardized approach to managing file storage operations across different backends. Each data connector includes a default file connector, and custom implementations can be added as needed by extending the FileConnector class present in Core.

Key methods of FileConnector:

• uploadFileContent(fileTag, dataType, binary): Uploads binary data to storage.
• downloadFileContent(fileTag, dataType): Retrieves binary data using a file tag.
• deleteFileContent(fileTag, dataType): Deletes files from storage.

Configuration properties, such as file paths or credentials, can be accessed via the getConfig(key) method. Custom file connectors need to be registered in the FileConnectorsRegistry. For example:

FileConnectorsRegistry.register(
  'customFileConnector',
  (configGetter) => new CustomFileConnector(configGetter)
);

How to Use File Connectors

To use a file connector, you need to specify the fileConnector property in the oConfig file, within the configuration of the data source. When no file connector is specified, it uses the one built-in the data source (eg: store files within the database).

{
   "data.<data_source_name>": {
      "host": "<host>",
      "user": "<user>",
      "password": "<password>",
      "database": "<database>",
      "schema": "<schema>",
      "fileConnector": "<file_connector_name>"
   }
}

---

Local Disk File Connector

The local disk file connector stores files directly on the server’s file system.

info

For retro-compatibility, if the filePath property is set in the data connector configuration, the local disk file connector is used even though not explicitly defined via the fileConnector property.

To use this connector, set the fileConnector property to fileSystem and add a filePath property to the data connector configuration to define where on the disk the files will be stored.

{
    "data.<data_source_name>": {
       ... // other data source properties
       "fileConnector": "fileSystem", 
       "filePath": "<file path>"
    }
}

---

Azure Blob Storage

To store files in Azure Blob Storage instead of the default file storage location, you need to update the oConfig file with specific properties.

Steps to Enable Azure Blob Storage

1. Add the fileConnector Property

   • To change the file storage from the default location to Azure Blob Storage, add the fileConnector property and set it to "azureBlobStorage" in the oConfig file.

2. Add the azure Object

   • Along with the fileConnector property, add the azure object. This object contains the authentication type (authType) and the necessary properties required for Azure Blob Storage.

Configuration Example

Here is an example configuration for the oConfig file:

{
   "data.<data_source_name>": {
      ... // other data source properties
      "fileConnector": "azureBlobStorage", 
      "azure": {
         "authType": "<authType> SAS, SASToken or SPN", 
         "blobUrl": "<blob-url>", 
         "container": "<container-name>", 
         "accountName": "<account-name>", 
         "accountKey": "<account-key>", 
         "tenantId": "<tenant-id>", 
         "clientId": "<client-id>", 
         "clientSecret": "<client-secret>", 
         "sasToken": "<sas-token>", 
         "pathFromRoot": "<optional-path-from-root>"
      }
   }
}

Below is a description of the properties required for Azure Blob Storage:

Parameter	Auth Type	Description
authType	All	The authentication type. Options: "SAS", "SPN", or "SASToken".
blobUrl	All	The URL of the Azure Blob Storage endpoint (e.g., emulator or Azure URL).
container	All	The name of the blob container where files will be stored.
accountName	SAS	The name of the Azure Storage account.
accountKey	SAS	The access key for the Azure Storage account.
sasToken	SASToken	The SAS token for accessing the blob container.
tenantId	SPN	The Azure Active Directory Tenant ID of the service principal.
clientId	SPN	The Client ID of the service principal.
clientSecret	SPN	The Client Secret of the service principal.
pathFromRoot	All	Optional. Specifies a subfolder within the container for file storage.