Skip to content

Support the DISTINCT set quantifier in aggregate function calls #4499

Description

@robert-brunel

The grammar admits an {ALL|DISTINCT} set quantifier on every aggregate function call, but semantic analysis currently rejects DISTINCT:

SELECT ARRAY_AGG(DISTINCT val) FROM T1
SELECT SUM(DISTINCT val) FROM T1
-- ERROR: aggregator DISTINCT is not supported

The rejection is in ExpressionVisitor.visitAggregateWindowedFunction() and applies uniformly to ARRAY_AGG(), SUM(), AVG(), MIN(), MAX(), COUNT() and GROUP_CONCAT().

What is needed:

  • De-duplicate the argument values within each group before they reach the aggregate function. For the scalar aggregates this changes the result of SUM(), AVG() and COUNT(); for MIN() and MAX() it is a no-op, so those could short-circuit to the plain form.
  • Decide where the de-duplication happens. For streaming aggregation, if the input is already sorted on the argument, it can drop adjacent duplicates cheaply. One approach is to generate appropriate ordering requirements for the inputs.

Related:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions