pip install -e fails to isolate build environment during metadata generation - namespace collision causes import error.

[f3flight]

Description

In the company I work at, I ran into the following issue:
I have a library A, which uses hatchling and an internal versioning plugin for it as build dependencies. This internal plugin C uses the following namespace layout "COMPANY.hatch.version"
I have an app B, which uses other internal libraries which also use "COMPANY. ..." namespace. Package A is not a dependency of app B.

What I have ran into is that the following sequence of commands results in a failure:

1. create & activate a venv
2. pip install -e B
3. pip install -e A ----> this fails with ModuleNotFoundError: No module named 'COMPANY.hatch' and error: metadata-generation-failed`

The following works:

1. create & activate a venv
2. pip install -e A
3. pip install -e B

Using PIP_NO_CLEAN=1 I have verified that /tmp/pip-build-env-xxxx/overlay contains package C and "COMPANY.hatch.version" module is present there. However it seems that metadata generation is not correctly isolated, and the process detects "COMPANY" top-level namespace in the current venv, and proceeds to import from there, and therefore fails to use the same namespace from the overlay.

It's a little troublesome for me to provide all the artifacts for a reproduction but I hope the problem is clear enough and whoever is interested in improving pip can create 3 small packages to imitate this situation.

Expected behavior

pip install -e B does not break consequent pip install -e A

pip version

25.2

Python version

3.12

OS

Linux

How to Reproduce

1. create a sample hatchling version plugin library, call it C, and make its code have namespace "somename.C.*"
2. create library A which uses hatchling and "C" as build dependency
3. create library B which uses "somename.B.*" as namespace
4. create a new venv and activate it
5. pip install -e B
6. pip install -e A ---> will fail due to import error of "somename.C"

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[sepehr-rs]

Hi @f3flight, thanks for taking the time to report this issue and provide detailed context!
I was able to reproduce the behavior, though I also noticed that running pip install -e A followed by pip install -e B didn’t work in my environment either, so the root cause might be a bit different or more complex than it appears.

Please note that it may take some time for a maintainer to review this in depth, as the pip maintainers are all volunteers. Thanks for your patience and for helping improve pip!

[pfmoore]

Hi @f3flight - the way namespace packages work, parts of the namespace from different locations should get merged. While I don’t have time to try your example right now, this doesn’t seem to be happening here.

That suggests that this may be a problem with hatch’s implementation of editable mode, so you should probably reach out to the hatch project. I believe hatch uses the editables library (of which I am the author) to implement editable installs, and the option they are using doesn’t support this aspect of namespace packages. If I’m correct, they would need to have an option to use editables other mode of operation to support this use case.

As I say, I don’t have time to try this myself, so the above is speculation. The hatch project should be able to confirm or disprove this theory.

[ofek]

Hatchling supports multiple modes for development installations. The mechanism I think you're referencing is not currently supported after taking a quick look at the logic. PRs always welcome!

I haven't read the current issue in full but folks should be aware of namespace packaging issues pfmoore/editables#34

[pfmoore]

I was referring to the add_to_path mechanism, which appears (from what I understand of the linked docs) to be the default mode used by hatch. So I'd expect namespace packages to work correctly in that case.

@f3flight can you provide a more detailed explanation of how to reproduce the issue? I don't know what "create a sample hatchling version plugin library, call it C, and make its code have namespace "somename.C.*"" means, so I was unable to even begin to reproduce the issue.

I will note that this is almost certainly not an issue with pip, though.