Connectors
Amazon S3
Configure Amazon S3 and S3-compatible providers for Beam transfers.
The S3 connector is a client SDK provider config. It signs source reads and destination writes in the SDK process, then gives BeamCore prepared URLs/routes for workers to execute.
Workers never receive AWS credentials.
TypeScript
import {
BeamClient,
R2ProviderConfig,
S3CompatibleProviderConfig,
S3ProviderConfig,
} from "@beam-network/sdk";
const beam = new BeamClient({ apiKey: process.env.BEAM_API_KEY! });
const transfer = await beam.createTransfer({
sources: [
S3ProviderConfig.create({
bucket: "source-bucket",
key: "datasets/2026/data.parquet",
region: "us-east-1",
access_key_id: process.env.AWS_ACCESS_KEY_ID!,
secret_access_key: process.env.AWS_SECRET_ACCESS_KEY!,
}),
],
destinations: [
R2ProviderConfig.create({
bucket: "dest-bucket",
key: "datasets/2026/data.parquet",
account_id: process.env.R2_ACCOUNT_ID,
access_key_id: process.env.R2_ACCESS_KEY_ID!,
secret_access_key: process.env.R2_SECRET_ACCESS_KEY!,
}),
],
name: "s3-to-r2",
});
console.log(transfer.transfer_id);S3-Compatible Providers
Use S3CompatibleProviderConfig for Wasabi, MinIO, Backblaze B2 S3 API, DigitalOcean Spaces, or any provider that needs a custom endpoint.
await beam.prepareProviderTransfer({
sources: [
S3CompatibleProviderConfig.create({
provider: "wasabi",
bucket: "my-bucket",
key: "input/file.bin",
region: "us-east-1",
endpoint_url: "https://s3.us-east-1.wasabisys.com",
access_key_id: process.env.WASABI_ACCESS_KEY_ID!,
secret_access_key: process.env.WASABI_SECRET_ACCESS_KEY!,
}),
],
destinations: [
S3CompatibleProviderConfig.create({
provider: "minio",
bucket: "archive",
key: "file.bin",
endpoint_url: "https://minio.example.com",
force_path_style: true,
access_key_id: process.env.MINIO_ACCESS_KEY_ID!,
secret_access_key: process.env.MINIO_SECRET_ACCESS_KEY!,
}),
],
name: "S3-compatible transfer",
});Python
from beam_network_sdk.models import S3ProviderDestination, S3ProviderSource
source = S3ProviderSource(
bucket="source-bucket",
key="datasets/2026/data.parquet",
region="us-east-1",
access_key_id="...",
secret_access_key="...",
)
destination = S3ProviderDestination(
bucket="dest-bucket",
key="datasets/2026/data.parquet",
region="us-east-1",
access_key_id="...",
secret_access_key="...",
)IAM Permissions
For a source bucket, the SDK needs read permission:
{
"Effect": "Allow",
"Action": ["s3:GetObject", "s3:HeadObject"],
"Resource": "arn:aws:s3:::my-bucket/*"
}For a destination bucket, the SDK needs write and multipart permissions:
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:CreateMultipartUpload", "s3:CompleteMultipartUpload", "s3:ListMultipartUploadParts", "s3:AbortMultipartUpload"],
"Resource": "arn:aws:s3:::my-bucket/*"
}