Add INLINE + ARROW_STREAM format support for analytics plugin#245
Open
jamesbroadhead wants to merge 2 commits intodatabricks:mainfrom
Open
Add INLINE + ARROW_STREAM format support for analytics plugin#245jamesbroadhead wants to merge 2 commits intodatabricks:mainfrom
jamesbroadhead wants to merge 2 commits intodatabricks:mainfrom
Conversation
Some serverless warehouses only support ARROW_STREAM with INLINE disposition, but the analytics plugin only offered JSON_ARRAY (INLINE) and ARROW_STREAM (EXTERNAL_LINKS). This adds a new "ARROW_STREAM" format option that uses INLINE disposition, making the plugin compatible with these warehouses. Fixes databricks#242
Tests verify: - ARROW_STREAM format passes INLINE disposition + ARROW_STREAM format - ARROW format passes EXTERNAL_LINKS disposition + ARROW_STREAM format - Default JSON format does not pass disposition or format overrides
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ARROW_STREAMwithINLINEdisposition, but the analytics plugin only offeredJSON_ARRAY(INLINE) andARROW_STREAM(EXTERNAL_LINKS)"ARROW_STREAM"format option that usesINLINEdisposition, making the plugin compatible with these warehousesAnalyticsFormattype to include"ARROW_STREAM"Test plan
useAnalyticsQuerywithformat: "ARROW_STREAM"returns results"JSON"and"ARROW"formats are unaffectedFixes #242