AWS S3 Node

AWS S3 Node

The AWS S3 node periodically polls data from an AWS S3 bucket, outputting payloads at the interval configured. It will pull data stored in your AWS S3 bucket onto the Rayven Platform.

Adding an AWS S3 node to your workflow

1. First, select your desired solution and navigate to Rayven Workflow.
2. Select ‘Inputs’ from the left-hand panel.
3. Find the AWS S3 node and drag it onto the canvas.
4. Double click on the AWS S3 node to open its configuration window.

Configuring an AWS S3 node

1. Give your node a name. Choose something simple that clearly explains its purpose. You must pick a name that has three or more characters.
2. Under Bucket Name, enter the name of the relevant bucket on AWS S3.
3. Enter the Access Key ID and Secret Key as configured on your AWS S3 account.
4. Enter the Folder Name within the bucket that contains the relevant data files.
5. Enter the AWS Region for your account. 
6. Enter the Interval between File Downloads to determine the frequency of data retrieval. This interval is measured in minutes and must be a numerical value greater than 0.
7. Under Payload Format, select the format of your AWS S3 files from the drop-down menu. The Rayven AWS S3 node supports JSON, JSON Array, CSV, CSV(no header row), XML and string formats.
   

Advanced Features

1. Enter a Payload Device ID Field Name to specify the JSON key where extracted Device ID information should appear. If you leave this field blank, payloads will have the device __none__. You will be able to access them via the debug node. 
2. Use Filter by File Name to define the files read from an AWS S3 folder. The Rayven AWS S3 node supports the use of wild cards (*). 
3. Select Variable Name for File Name if you want to set the JSON key that will hold the file name as the value used in the workflow. 
4. Check the Remove Extension from File Name to remove the extension from a file name when Variable Name for File Name is selected. (e.g. to remove .csv, .txt, etc).
   

Payload Date Settings

You can configure data timestamps based on when data arrives from a device. You will need to define the following fields:

1. Under Date Field Name, enter the name of the date field in the device payload.
2. Define the Date Format, for example, dd/MM/yyyy or HH:mm:ss. If data is in a 24-hour format, make sure you capitalise the hour letters (i.e. HH:mm:ss).

Activation

Once you have configured the node, click 'activate' and then 'save'. The node is now ready to send data into the workflow.