fix(optimizer): preserve UNNEST struct-field columns - #7979
fix(optimizer): preserve UNNEST struct-field columns#7979fivetran-kwoodbeck wants to merge 4 commits into
Conversation
SQLGlot Integration Test Results✅ All tests passedComparing:
Overallmain: 192414 total, 153545 passed (pass rate: 79.8%) sqlglot:optimizer/fix-numbering-unnest-canonicalize-internal-names: 180220 total, 142393 passed (pass rate: 79.0%) Transitions: Dialect pair changes: 0 previous results not found, 3 current results not found ✅ All tests passed |
e932791 to
8a47ac3
Compare
| and (element_type := seq_get(src.expressions[0].type.expressions, 0)) | ||
| ): | ||
| preserve_names = preserve_names or element_type.is_type(exp.DataType.Type.STRUCT) | ||
|
|
There was a problem hiding this comment.
Input (bq):
SELECT s.sid FROM UNNEST([STRUCT('a' AS sid)]) AS s;
SELECT z.sid FROM UNNEST([STRUCT('a' AS sid)]) AS z
main (output for both queries):
SELECT `_c0`.`sid` AS `sid` FROM UNNEST([STRUCT('a' AS `sid`)]) AS `_c0`
PR (output):
SELECT `s`.`sid` AS `sid` FROM UNNEST([STRUCT('a' AS `sid`)]) AS `s`
SELECT `z`.`sid` AS `sid` FROM UNNEST([STRUCT('a' AS `sid`)]) AS `z`
The granularity of the change should be column-level rather than source-level: only the struct field names need preserving, but the source-level flag also preserves the UNNEST's element alias (s/z above), which leaks them into the canonical form. Since both inputs are the same query, they should keep producing the same canonical output, as on main. cc: @georgesittas
There was a problem hiding this comment.
Yeah, and we should also add a test that does something like:
SELECT s FROM UNNEST([STRUCT('a' AS sid)]) AS s;Because the source name now acts as both a source and a top-level column (i.e., part of the contract).
canonicalize_internal_nameswas producing invalid SQL for struct field columns of anUNNEST. An UNNEST of structs exposes struct field names as physical columns, but the rule was treating them as internal handles.Sample Input
Previous Output, invalid SQL: references non existent
_c0Fixed Output: field name
sidpreservedFix
In canonicalize_internal_names, added a
preserve_namesflag that treats an UNNEST + structs field column as physical. This is the same wayis_base_sourcecolumns are already preserved. Scalar UNNEST element aliases are renameable definitions, so they still canonicalize like previously.Behavior
_cN_cN