feat: add array_scale scalar function#22466
Conversation
github-actions
Bot
added
documentation
|
|
||
| let mut value_builder = Float64Array::builder(values.len()); | ||
| let mut new_offsets = OffsetBufferBuilder::<O>::new(list_array.len()); | ||
| let mut row_nulls = NullBufferBuilder::new(list_array.len()); |
There was a problem hiding this comment.
Don't need to calculate per row nulls, can just use NullBuffer::union
let nulls = NullBuffer::union(list_array.nulls(), scalar_array.nulls());There was a problem hiding this comment.
Done. Replaced the NullBufferBuilder with let row_nulls = NullBuffer::union(list_array.nulls(), scalar_array.nulls()) computed once before the loop, then passed directly into GenericListArray::try_new. The is_null check inside the loop is kept since it's still load-bearing for the offset buffer construction (we need to push zero-length offsets for null rows).
There was a problem hiding this comment.
We can use the new row_nulls for the null check inside the loop now instead of still querying list + scalar separately
e.g.
row_nulls.is_some_and(|nb| nb.is_null(i))There was a problem hiding this comment.
Done. Loop check now uses row_nulls.as_ref().is_some_and(|nb| nb.is_null(row)) instead of querying list_array and scalar_array separately. Same semantics (NullBuffer::union ORs the two null buffers), one source of truth.
crm26
force-pushed
the
feat/array-scale
branch
2 times, most recently
from
a29be46 to
bf052a9
Compare
Adds `array_scale(array, scalar)` returning a new array with each element multiplied by a scalar. Aliased as `list_scale`. Part of the per-function split sequence on tracking issue apache#21536, following the pattern of the already-merged PRs in this series. Semantics: - NULL row in array -> NULL row out - NULL element at position i in array -> NULL element at i out (per-element propagation) - NULL scalar -> NULL row out (whole-row, because the scalar applies uniformly to every element; the entire operation is undefined) - Empty array -> empty array First argument is List/LargeList/FixedSizeList of any numeric type. Second argument is a numeric scalar. Both coerce to Float64. List-like inputs follow the same widening rules as the binary-op siblings: LargeList wins, FixedSizeList coerces to List.
2 participants
Which issue does this PR close?
Partial of #21536 —
array_scale(the list+scalar arithmetic function in the vector math series).Rationale for this change
Continues the per-function split requested by @alamb on #21536. Three sibling PRs already merged:
cosine_distance(#21542),inner_product(#21861),array_normalize(#22013).array_addis in flight as #22459 by @SubhamSinghal.Adds element-wise scalar multiplication for numeric arrays, returning a list of the same shape. Aliased as
list_scaleto match thearray_X/list_Xprecedent in this crate.What changes are included in this PR?
array_scale(array, scalar)indatafusion/functions-nested/src/array_scale.rsdatafusion/functions-nested/src/lib.rsdatafusion/sqllogictest/test_files/array_scale.sltdocs/source/user-guide/sql/scalar_functions.mdSignature: first arg
List/LargeList/FixedSizeList<numeric>, second arg numeric scalar. Both coerce toFloat64. Same list-widening rules as the binary-op siblings.NULL semantics:
Builders: uses `OffsetBufferBuilder` + `NullBufferBuilder` per the pattern adopted in the round-1 review of #22013.
Are these changes tested?
Yes. `array_scale.slt` covers:
Are there any user-facing changes?
Yes — new SQL scalar function `array_scale(array, scalar)` and its alias `list_scale`. Documented in `docs/source/user-guide/sql/scalar_functions.md`.