Skip to content

Devalue uxarray to make it less expensive#1588

Draft
cmdupuis3 wants to merge 2 commits into
UXARRAY:mainfrom
cmdupuis3:cmd/devalue
Draft

Devalue uxarray to make it less expensive#1588
cmdupuis3 wants to merge 2 commits into
UXARRAY:mainfrom
cmdupuis3:cmd/devalue

Conversation

@cmdupuis3

@cmdupuis3 cmdupuis3 commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Closes #1583

Overview

This PR is to resolve suboptimal usage of .values throughout the repo, primarily by deferring to lazy xarray and dask operations. See the table of usage sites in Issue #1583.

PR Checklist

General

  • An issue is linked created and linked
  • Add appropriate labels
  • Filled out Overview and Expected Usage (if applicable) sections

Testing

  • Adequate tests are created if there is new functionality
  • Tests cover all possible logical paths in your function
  • Tests are not too basic (such as simply calling a function and nothing else)

Documentation

  • Docstrings have been added to all new functions
  • Docstrings have updated with any function changes
  • Internal functions have a preceding underscore (_) and have been added to docs/internal_api/index.rst
  • User functions have been added to docs/user_api/index.rst

Examples

  • Any new notebook examples added to docs/examples/ folder
  • Clear the output of all cells before committing
  • New notebook files added to docs/examples.rst toctree
  • New notebook files added to new entry in docs/gallery.yml with appropriate thumbnail photo in docs/_static/thumbnails/

@cmdupuis3 cmdupuis3 self-assigned this Jul 18, 2026
@cmdupuis3 cmdupuis3 added scalability Related to scalability & performance efforts run-benchmark Run ASV benchmark workflow labels Jul 18, 2026
@github-actions

github-actions Bot commented Jul 18, 2026

Copy link
Copy Markdown

ASV Benchmarking

Benchmark Comparison Results

Benchmarks that have improved:

Change Before [b0487c4] After [5ea2d9e] Ratio Benchmark (Parameter)
- 581M 391M 0.67 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
- 700M 390M 0.56 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
- 498M 384M 0.77 mpas_ocean.Gradient.peakmem_gradient('480km')
- 4.80±0.03ms 4.30±0.08ms 0.9 mpas_ocean.RemapDownsample.time_nearest_neighbor_remapping

Benchmarks that have stayed the same:

Change Before [b0487c4] After [5ea2d9e] Ratio Benchmark (Parameter)
10.2±0.1μs 9.93±0.09μs 0.97 bench_connectivity.Connectivity.time_edge_face('120km')
10.2±0.2μs 10.3±0.1μs 1.02 bench_connectivity.Connectivity.time_edge_face('480km')
10.1±0.1μs 10.1±0.06μs 1.00 bench_connectivity.Connectivity.time_edge_node('120km')
10.1±0.1μs 10.1±0.01μs 0.99 bench_connectivity.Connectivity.time_edge_node('480km')
10.0±0.1μs 9.99±0.2μs 1.00 bench_connectivity.Connectivity.time_face_edge('120km')
10.3±0.02μs 10.4±0.1μs 1.01 bench_connectivity.Connectivity.time_face_edge('480km')
9.93±0.04μs 9.85±0.05μs 0.99 bench_connectivity.Connectivity.time_face_face('120km')
10.2±0.04μs 10.2±0.1μs 1.00 bench_connectivity.Connectivity.time_face_face('480km')
20.1±0.2μs 20.4±0.2μs 1.01 bench_connectivity.Connectivity.time_face_node('120km')
20.6±0.3μs 20.9±0.4μs 1.01 bench_connectivity.Connectivity.time_face_node('480km')
10.1±0.09μs 10.4±0.04μs 1.04 bench_connectivity.Connectivity.time_node_edge('120km')
10.1±0.2μs 10.3±0.1μs 1.02 bench_connectivity.Connectivity.time_node_edge('480km')
9.94±0.05μs 10.4±0.2μs 1.05 bench_connectivity.Connectivity.time_node_face('120km')
10.5±0.2μs 10.3±0.1μs 0.99 bench_connectivity.Connectivity.time_node_face('480km')
389M 389M 1.00 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
419M 421M 1.01 face_bounds.FaceBounds.peakmem_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
12.9±0.07ms 12.8±0.04ms 0.99 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/mpas/QU/oQU480.231010.nc'))
3.40±0.06ms 3.37±0.06ms 0.99 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/scrip/outCSne8/outCSne8.nc'))
17.4±0.08ms 17.5±0.05ms 1.01 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/geoflow-small/grid.nc'))
2.00±0.01ms 2.02±0.04ms 1.01 face_bounds.FaceBounds.time_face_bounds(PosixPath('/home/runner/work/uxarray/uxarray/test/meshfiles/ugrid/quad-hexagon/grid.nc'))
1.03±0.02s 1.03±0.04s 1.00 import.Imports.timeraw_import_uxarray
901±20ns 908±4ns 1.01 mpas_ocean.CheckNorm.time_check_norm('120km')
870±10ns 850±5ns 0.98 mpas_ocean.CheckNorm.time_check_norm('480km')
735±10ms 729±30ms 0.99 mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('120km')
45.2±0.2ms 45.2±0.7ms 1.00 mpas_ocean.ConnectivityConstruction.time_face_face_connectivity('480km')
634±6μs 639±10μs 1.01 mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('120km')
582±7μs 567±10μs 0.97 mpas_ocean.ConnectivityConstruction.time_n_nodes_per_face('480km')
5.07±0.01ms 5.09±0.06ms 1.00 mpas_ocean.ConstructFaceLatLon.time_cartesian_averaging('120km')
3.58±0.02ms 3.53±0.07ms 0.99 mpas_ocean.ConstructFaceLatLon.time_cartesian_averaging('480km')
3.30±0.01s 3.31±0.01s 1.00 mpas_ocean.ConstructFaceLatLon.time_welzl('120km')
214±0.4ms 212±2ms 0.99 mpas_ocean.ConstructFaceLatLon.time_welzl('480km')
19.7±0.01ms 19.7±0.05ms 1.00 mpas_ocean.ConstructTreeStructures.time_ball_tree('120km')
1.15±0.02ms 1.13±0.06ms 0.98 mpas_ocean.ConstructTreeStructures.time_ball_tree('480km')
10.7±0.03ms 10.7±0.06ms 1.00 mpas_ocean.ConstructTreeStructures.time_kd_tree('120km')
780±10μs 752±40μs 0.96 mpas_ocean.ConstructTreeStructures.time_kd_tree('480km')
679±5ms 674±30ms 0.99 mpas_ocean.CrossSections.time_const_lat('120km', 1)
343±5ms 340±20ms 0.99 mpas_ocean.CrossSections.time_const_lat('120km', 2)
172±0.9ms 170±6ms 0.99 mpas_ocean.CrossSections.time_const_lat('120km', 4)
491±4ms 503±20ms 1.02 mpas_ocean.CrossSections.time_const_lat('480km', 1)
251±4ms 252±7ms 1.00 mpas_ocean.CrossSections.time_const_lat('480km', 2)
131±1ms 131±4ms 1.00 mpas_ocean.CrossSections.time_const_lat('480km', 4)
22.2±0.1ms 22.4±0.2ms 1.01 mpas_ocean.DualMesh.time_dual_mesh_construction('120km')
2.65±0.03ms 2.63±0.09ms 0.99 mpas_ocean.DualMesh.time_dual_mesh_construction('480km')
847±3ms 844±5ms 1.00 mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', False)
50.4±0.8ms 51.1±1ms 1.01 mpas_ocean.GeoDataFrame.time_to_geodataframe('120km', True)
73.9±1ms 73.3±0.4ms 0.99 mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', False)
5.67±0.1ms 5.70±0.3ms 1.00 mpas_ocean.GeoDataFrame.time_to_geodataframe('480km', True)
404M 404M 1.00 mpas_ocean.Gradient.peakmem_gradient('120km')
167±2ms 167±2ms 1.00 mpas_ocean.Gradient.time_gradient('120km')
11.5±0.04ms 11.8±0.2ms 1.03 mpas_ocean.Gradient.time_gradient('480km')
189±2μs 191±2μs 1.01 mpas_ocean.HoleEdgeIndices.time_construct_hole_edge_indices('120km')
87.1±0.6μs 85.6±0.8μs 0.98 mpas_ocean.HoleEdgeIndices.time_construct_hole_edge_indices('480km')
351M 351M 1.00 mpas_ocean.Integrate.peakmem_integrate('120km')
331M 331M 1.00 mpas_ocean.Integrate.peakmem_integrate('480km')
188±2ms 189±3ms 1.01 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'exclude')
189±2ms 189±1ms 1.00 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'include')
191±2ms 191±3ms 1.00 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('120km', 'split')
14.3±0.1ms 14.3±0.5ms 1.00 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'exclude')
14.4±0.1ms 14.7±0.7ms 1.02 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'include')
14.3±0.08ms 14.4±0.6ms 1.01 mpas_ocean.MatplotlibConversion.time_dataarray_to_polycollection('480km', 'split')
284±2μs 282±3μs 0.99 mpas_ocean.PointInPolygon.time_face_search_lonlat('120km')
283±2μs 286±0.7μs 1.01 mpas_ocean.PointInPolygon.time_face_search_lonlat('480km')
266±2μs 266±0.4μs 1.00 mpas_ocean.PointInPolygon.time_face_search_xyz('120km')
267±2μs 268±1μs 1.00 mpas_ocean.PointInPolygon.time_face_search_xyz('480km')
214±0.4ms 215±1ms 1.00 mpas_ocean.RemapDownsample.time_bilinear_remapping
232±0.5ms 231±0.9ms 0.99 mpas_ocean.RemapDownsample.time_inverse_distance_weighted_remapping
1.19±0.01s 1.19±0.01s 1.00 mpas_ocean.RemapUpsample.time_bilinear_remapping
37.3±0.6ms 37.6±0.4ms 1.01 mpas_ocean.RemapUpsample.time_inverse_distance_weighted_remapping
9.17±0.2ms 9.38±0.2ms 1.02 mpas_ocean.RemapUpsample.time_nearest_neighbor_remapping
29.4±0.6ms 29.9±0.6ms 1.02 mpas_ocean.ZonalAverage.time_zonal_average('120km')
6.32±0.06ms 6.84±0.2ms 1.08 mpas_ocean.ZonalAverage.time_zonal_average('480km')
326M 326M 1.00 quad_hexagon.QuadHexagon.peakmem_open_dataset
325M 326M 1.00 quad_hexagon.QuadHexagon.peakmem_open_grid
6.61±0.1ms 7.45±0.8ms ~1.13 quad_hexagon.QuadHexagon.time_open_dataset
5.52±0.2ms 6.48±0.3ms ~1.17 quad_hexagon.QuadHexagon.time_open_grid

Benchmarks that have got worse:

Change Before [b0487c4] After [5ea2d9e] Ratio Benchmark (Parameter)
+ 179±1μs 643±10μs 3.6 mpas_ocean.Integrate.time_integrate('120km')
+ 159±1μs 594±8μs 3.74 mpas_ocean.Integrate.time_integrate('480km')

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-benchmark Run ASV benchmark workflow scalability Related to scalability & performance efforts

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Census of ".values" and cleanup

1 participant