Skip to content

Analytics plugin incompatible with ARROW_STREAM-only warehouses #242

@jamesbroadhead

Description

@jamesbroadhead

Summary

AppKit's analytics plugin defaults to disposition: "INLINE" + format: "JSON_ARRAY" for SQL queries. Some serverless warehouses only support ARROW_STREAM for inline disposition, causing all analytics queries to fail at runtime.

The ARROW code path (format: "ARROW") uses disposition: "EXTERNAL_LINKS", which some warehouses also reject with NOT_IMPLEMENTED: ExternalLinks disposition is not yet implemented.

This means certain warehouses are fundamentally incompatible with AppKit's analytics plugin — there is no format/disposition combination that works.

Reproduction

  1. Deploy an AppKit app using a serverless SQL warehouse that enforces ARROW_STREAM inline (e.g. warehouse 000000000000000d on appsdev)
  2. Use useAnalyticsQuery with default JSON format → fails with INVALID_PARAMETER_VALUE: Inline disposition only supports ARROW_STREAM format
  3. Use useAnalyticsQuery with format: "ARROW" → fails with NOT_IMPLEMENTED: ExternalLinks disposition is not yet implemented
  4. Use BarChart with format="arrow" → same EXTERNAL_LINKS error

Root cause

@databricks/appkit/dist/connectors/sql-warehouse/defaults.js:

const executeStatementDefaults = {
    disposition: "INLINE",
    format: "JSON_ARRAY",
};

ARROW path in analytics.js line 104-108:

formatParameters: {
    disposition: "EXTERNAL_LINKS",
    format: "ARROW_STREAM"
}

Neither path supports INLINE + ARROW_STREAM, which is the only combination some warehouses accept.

Suggested fix

Add an INLINE + ARROW_STREAM code path:

  1. Detect when JSON_ARRAY is rejected and auto-retry with ARROW_STREAM inline
  2. Or add a config option: analytics({ format: "ARROW_STREAM" })
  3. Or probe the warehouse at startup with a SELECT 1 test query to determine supported formats

Environment

  • @databricks/appkit v0.14.1
  • Warehouse: serverless PRO on AWS dev workspace

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions