feat(uv): Add a clean-sheet dependency implementation #650

arrdem · 2025-09-25T18:26:22Z

An implementation of pip based on consuming PEP-751 [1] like lockfiles. Specifically uv lockfiles, which contain internal dependency graph information that the PEP-751 specification labels optional.

Follows in the footsteps of rules_js's pnpm support by consuming a lockfile which contains enough information to produce materialize dependencies without performing any repository time operations which could be platform dependent.

Features

Supports cross-platform builds of wheels
Supports hermetic source builds of wheels
Automatically handles dependency cycles
Creates unified pip hubs which span virtualenv/dependency solution boundaries
Pip library targets from deactivated venvs are incompatible
Platform constraints on pip libraries do not prevent the creation of a library target

Example

# MODULE.bazel content
uv = use_extensioin("@aspect_rules_py//uv:extesion.bzl", "uv")
uv.declare_hub(hub_name = "pip")
    
uv.declare_venv(hub_name = "pip", venv_name = "a")
uv.lockfile(hub_name = "pip", venv_name = "a", lockfile = "third_party/py/venvs/uv-a.lock")

uv.declare_venv(hub_name = "pip", venv_name = "b")
uv.lockfile(hub_name = "pip", venv_name = "b", lockfile = "third_party/py/venvs/uv-b.toml")

use_repo(pip, "pip")

# BUILD.bazel content

py_venv_binary(
  name = "foo",
  srcs = [
    "foo.py",
  ],
  main = "foo.py",
  deps = [
    "@pip//cowsay", # Pull cowsay from the configured venv
  ],
  venv = "a", # Configure the default venv to be "a"; may be overriden at the CLI
)

The active venv state can be overriden at the cli by specifying --'@pip//venv=b' here for instance, or by using transitions to(re) set that same flag.

Appendix

[1] https://peps.python.org/pep-0751/
[2] https://peps.python.org/pep-0751/#locking-build-requirements-for-sdists

Changes are visible to end-users: yes

Searched for relevant documentation and updated as needed: yes/no
Breaking change (forces users to change their own code or config): yes/no
Suggested release notes appear below: yes/no

To do list

Test plan

Manually test flipping the venv command line flag
Manually test flipping the venv transition attr
Create py_venv_tests covering that different versions of the same package can be concurrently configured via different venvs
Create a py_venv_test covering that Airflow or another package with dependency cycles can be provisioned
Create a py_venv_binary embedded in and transitioned for a Linux OCI container across arch boundaries

aspect-workflows · 2025-09-25T19:56:45Z

test-os:linux-bzl:8 (Test)

⚠️ Buildkite build #821 failed.

Failed tests (2)

//py/tests/py_venv_conflict:validate_import_roots [k8-fastbuild-ST-dc485d03186b]       🔗
//py/tests/py_venv_image_layer:my_app_amd64_layers_test [k8-fastbuild]                 🔗

💡 To reproduce the test failures, run

bazel test //py/tests/py_venv_conflict:validate_import_roots //py/tests/py_venv_image_layer:my_app_amd64_layers_test

test-os:linux-bzl:latest (Test)

⚠️ Buildkite build #821 failed.

Failed tests (2)

//py/tests/py_venv_conflict:validate_import_roots [k8-fastbuild-ST-dc485d03186b]       🔗
//py/tests/py_venv_image_layer:my_app_amd64_layers_test [k8-fastbuild]                 🔗

💡 To reproduce the test failures, run

bazel test //py/tests/py_venv_conflict:validate_import_roots //py/tests/py_venv_image_layer:my_app_amd64_layers_test

test-os:linux-bzl:8 (Test)

e2e

⚠️ Buildkite build #821 failed.

test-os:linux-bzl:latest (Test)

e2e

⚠️ Buildkite build #821 failed.

test-os:linux-bzl:8 (Test)

examples/uv_pip_compile

All tests were cache hits

1 test (100.0%) was fully cached saving 335ms.

test-os:linux-bzl:latest (Test)

examples/uv_pip_compile

All tests were cache hits

1 test (100.0%) was fully cached saving 335ms.

pip/private/sdist_build/rule.bzl

dzbarsky · 2025-09-29T23:57:02Z

uv/private/constraints/abi/macro.bzl

+
+            for minor in MINORS:
+                selects.config_setting_group(
+                    name = "is_{}{}{}".format(interpreter, major, minor),


generally %s is faster than .format because it avoids a function call and has simpler (faster) arg conversion. Also if you have only a single substitution it helps even more because you don't even need to construct a tuple. Up to you where to use it, but at least for the functions in the extension that are called repeatedly I would strongly consider it. (Honestly I pretty much always use it unless I have a complex/multiline template with 4+ args and/or using an arg multiple times)

dzbarsky

since you said you love nits...i left a bunch of nits. feel free to apply or disregard as you see fit!

dzbarsky · 2025-09-29T23:57:52Z

uv/private/constraints/defs.bzl

+    # We loop up to the second-to-last item to ensure we always have a 'next' stage.
+    for i, current_stage in enumerate(stages):
+        selects.config_setting_group(
+            name = "{}".format(current_stage.name),


nit: name = current_stage.name :)

dzbarsky · 2025-09-29T23:58:33Z

uv/private/constraints/defs.bzl

+        selects.config_setting_group(
+            name = "{}".format(current_stage.name),
+            match_any = [
+                ":{}".format(current_stage.condition),


":" + current_Stage.condition or ":%s" % current_stage.condition are both faster and less chars :)

dzbarsky · 2025-09-30T00:01:47Z

uv/private/constraints/repository.bzl

+"""
+selects.config_setting_group(
+    name = "{}",
+    match_all = {},


it might be a bit clearer to put the brackets in the expression here, especially since I think the way you did it I think the trailing ] isn't indented correctly? Here's a similar thing I did with build_deps here

dzbarsky · 2025-09-30T00:04:07Z

uv/private/extension.bzl

+    # Collect all hubs, ensure we have no dupes
+    for mod in module_ctx.modules:
+        for hub in mod.tags.declare_hub:
+            hub_specs.setdefault(hub.hub_name, {})


I saw you use setdefault like this a bunch, I think you can tweak the pattern slightly to avoid repeated lookups:

hub_specs.setdefault(hub.hub_name, {})[mod.name] = 1

dzbarsky · 2025-09-30T00:07:04Z

uv/private/extension.bzl

+    problems = []
+    for hub_name, modules in hub_specs.items():
+        if len(modules.keys()) > 1:
+            problems.append(


I think you could combine this with the previous loop and make hub_specs store hub_name -> mod_name(just check if you already have an entry and add toproblems) instead of of hub_name -> mod_name -> 1`. I guess it would make it harder to handle 3 modules all using the same name, but meh? It would probably lead to simpler usage in the rest of the extension

pip/private/whl_install/repository.bzl

dzbarsky · 2025-09-30T00:41:30Z

uv/private/whl_install/repository.bzl

+load(":parse_whl_name.bzl", "parse_whl_name")
+
+def format_arms(d):
+    content = ["        \"{}\": \"{}\"".format(k, v) for k, v in d.items()]


maybe clearer to use single quotes on the outside and/or repr the k/v to avoid inner ones?

pip/private/whl_install/repository.bzl

pip/private/whl_install/rule.bzl

dzbarsky · 2025-09-30T00:44:41Z

uv/private/whl_install/rule.bzl

+    return [
+        # FIXME: Need to generate PyInfo here
+        DefaultInfo(
+            files = depset([


i guess you could create this depset once and reuse it below

arrdem · 2025-10-15T17:11:01Z

TODO: Can we rip the uv toolchain out?

arrdem · 2025-10-30T17:52:11Z

, For later, more of Ignas' fine code we can leverage https://github.com/bazel-contrib/rules_python/pull/2692/files, bazel-contrib/rules_python#2856

dzbarsky reviewed Sep 27, 2025

View reviewed changes

pip/private/sdist_build/rule.bzl Outdated Show resolved Hide resolved

dzbarsky reviewed Sep 29, 2025

View reviewed changes

dzbarsky reviewed Sep 30, 2025

View reviewed changes

arrdem marked this pull request as ready for review October 3, 2025 17:36

arrdem force-pushed the arrdem/feat-pip branch 2 times, most recently from 4c2da1c to 82e6fc8 Compare October 9, 2025 17:39

arrdem changed the title ~~feat(pip): Add a clean-sheet pip implementation~~ feat(uv): Add a clean-sheet dependency implementation Oct 9, 2025

arrdem force-pushed the arrdem/feat-pip branch 4 times, most recently from 6505698 to 95a9045 Compare October 15, 2025 03:28

arrdem force-pushed the arrdem/feat-pip branch 3 times, most recently from ab53104 to c8658ee Compare October 28, 2025 01:40

arrdem mentioned this pull request Oct 28, 2025

[FR]: Better support for the new repository cache #687

Closed

arrdem force-pushed the arrdem/feat-pip branch 2 times, most recently from 67f2f3b to 12139ec Compare October 28, 2025 20:23

feat(uv): Create a UV driver

cb06830

arrdem force-pushed the arrdem/feat-pip branch from 12139ec to cb06830 Compare October 28, 2025 20:25

aspect-marvin and others added 5 commits October 28, 2025 20:37

Pre-commit changes

3bf7721

Set a bunch of visibility specs to open for now

ffeb879

Pre-commit changes

fde3c4f

Switch over to @dzbarsky's toml2json

d17a28f

Pre-commit changes

5b16dc2

arrdem added 2 commits October 30, 2025 12:05

[NO TESTS] WIP

7556bba

Rip out the ladders for gte rules

31aca7f

Uh oh!

feat(uv): Add a clean-sheet dependency implementation #650

Are you sure you want to change the base?

feat(uv): Add a clean-sheet dependency implementation #650

Uh oh!

Conversation

arrdem commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Features

Example

Appendix

Changes are visible to end-users: yes

To do list

Test plan

Uh oh!

aspect-workflows bot commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

e2e

e2e

examples/uv_pip_compile

examples/uv_pip_compile

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dzbarsky left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arrdem commented Oct 15, 2025

Uh oh!

arrdem commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

arrdem commented Sep 25, 2025 •

edited

Loading

aspect-workflows bot commented Sep 25, 2025 •

edited

Loading

arrdem commented Oct 30, 2025 •

edited

Loading