Architecture

Security & Auth

How dataflows handles secrets, tokens, and permissions.

Token Management

Handling OAuth2 tokens is complex. dataflows abstracts this away completely.

The connections Table

Tokens are stored in the connections table.

  • Encryption: access_token and refresh_token are encrypted using AES-256-GCM before being written to the database.
  • Auto-Refresh: The engine automatically checks expires_at and refreshes the token before a workflow step runs.

Service Clients

We separate the "Application" (Service Client) from the "User" (Connection).

  • Service Client: Contains client_id and client_secret.
  • Connection: Contains the user's access_token.

Dependency Injection

Workflows never access secrets directly. The engine injects pre-authenticated clients into each step.

// ❌ BAD: Handling tokens manually
const token = await db.tokens.find(...)
await fetch('https://api.github.com', {
  headers: { Authorization: token }
})

// ✅ GOOD: Authenticated client injected by the engine
export const syncIssues = defineWorkflow({
  id: 'sync-github-issues',
  trigger: schedule.every('1h'),

  async run({ step, clients }) {
    // The engine resolves the connection, refreshes tokens
    // if needed, and hands you a ready-to-use client.
    return step.run('fetch-issues', () =>
      clients.github.listIssues({ repo: 'dataflows-io/app' })
    )
  }
})

Isolation

Each workflow execution runs in an isolated context.

  • Environment Variables: Secrets are injected as environment variables.
  • Network Isolation: Workflows can be restricted to specific egress domains.

Ready for automation that just works?

Tell us about your process. We will tell you if and how dataflows can help.