Skip to content

[Feature][task-flink] Submit Flink SQL through SQL Gateway #18691

Description

@macdoor

Feature proposal

What problem does this feature solve?

Add a new task type FLINK_SGW that submits SQL scripts to a Flink SQL Gateway via its JDBC driver (flink-sql-jdbc-driver), so batch SQL workloads can be scheduled by DolphinScheduler without a local Flink client installation.

Main capabilities:

  • Main SQL script source: inline script or resource-center file (SqlSourceType.SCRIPT / FILE)
  • Optional init script (same two sources), executed before the main script (e.g. SET pipeline.name=... and DDL)
  • Configurable statement separator (default ;), statements are split and submitted one by one
  • Leading line comments (--) and blank lines are stripped per statement so the Flink parser receives clean statements
  • JDBC connection properties are configurable; flinkJdbcUrl supports parameter placeholders (e.g. ${flink_sqlgateway_url})
  • Optional query-result row printing (maxPrintRows) to avoid log explosion

Is this a regression?

No, this is a new task plugin.

Additional context

The plugin has been running in an internal production environment (schedule-only batch workloads through a dedicated Flink SQL Gateway) since 2026-03, which is where the naming and layout motivation comes from.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions