Docs / Strand / connectors/aws-kinesis

AWS Kinesis Connector

Stream real-time data into and out of Amazon Kinesis Data Streams from your Strand workflows.

Prerequisites

You need AWS credentials with Kinesis permissions. Here's how to set them up:

1. Sign in to the AWS Management Console and navigate to IAM
2. Create a new IAM user or use an existing one
3. Attach a policy with the permissions listed in Required Permissions below
4. Under Security credentials, create an Access key
5. Copy the Access Key ID and Secret Access Key

Required Permissions

The IAM user or role must have the following permissions:

Tip: Scope permissions to specific stream ARNs. Use a dedicated IAM user with only the permissions your workflows need.

Connector Setup

Create an AWS Kinesis connector from the Connectors page.

Configuration Fields

Operations

Put Record

Write a single data record to a Kinesis stream.

The record payload is the incoming event payload (serialized to JSON). There is no separate Data field; shape the payload with an upstream node if needed.

Example: the event payload sent as the record

{
  "event": "order_placed",
  "order_id": "ORD-001",
  "customer": "Alice",
  "total": 149.99,
  "timestamp": "2026-02-18T14:30:00Z"
}

Get Records

Read records from a specific shard in a Kinesis stream.

Example: Read records from a shard

• Stream Name: order-events
• Shard ID: shardId-000000000000
• Limit: 50

Output

Put Record Output

{
  "success": true,
  "status": "sent",
  "data": {
    "stream": "order-events",
    "shard_id": "shardId-000000000001",
    "sequence_number": "49607689048506540625859353263605505338738884163461496834"
  },
  "service": "aws.kinesis",
  "operation": "put_record"
}

Key fields for subsequent nodes:

• {{ steps.node_a1b2c3d4.output_payload.data.shard_id }}: Shard the record was written to
• {{ steps.node_a1b2c3d4.output_payload.data.sequence_number }}: Unique sequence number for the record

Get Records Output

Each record is decoded from its JSON payload back into an object (records that aren't valid JSON are returned as {"raw": "..."}). status is received when records are returned, or empty otherwise.

{
  "success": true,
  "status": "received",
  "data": {
    "stream": "order-events",
    "shard_id": "shardId-000000000000",
    "records": [
      {"event": "order_placed", "order_id": "ORD-001"}
    ],
    "count": 1
  },
  "service": "aws.kinesis",
  "operation": "get_records"
}

Key fields for subsequent nodes:

• {{ steps.node_a1b2c3d4.output_payload.data.records }}: Array of decoded records
• {{ steps.node_a1b2c3d4.output_payload.data.count }}: Number of records returned

Errors

Example Workflow

1. Create Connector with your AWS credentials and region us-east-1 (or use auth_type: "role" to leverage the host's IAM role)
2. Put record when an order event arrives (the event payload is sent as the record):

• Stream Name: order-events
• Partition Key: {{ payload.customer_id }} (optional; defaults to a generated UUID)

1. Log confirmation to a downstream node:

• Reference: {{ steps.put_order.output_payload.data.sequence_number }}

Limitations

• Single record writes: Each Put Record operation writes one record. For batch writes, use multiple workflow nodes or a loop.
• Shard management: Shard splitting and merging must be managed through the AWS Console or CLI.
• Record size: Individual records are limited to 1 MB by AWS.
• Data retention: Kinesis retains data for 24 hours by default (up to 365 hours with extended retention enabled in AWS).
• Consumer tracking: Checkpoint management for consumers is not handled automatically. Use sequence numbers from output to track position.

← Back to Strand docs