Remove use of `python setup.py develop/install` in the project #2172

ndgrigorian · 2025-10-15T00:30:59Z

A warning that python setup.py develop is deprecated and will no longer be supported at the end of October, 2025 has been ongoing for some time when using dpctl build driver script, and can be seen in the CI

This PR proposes instead relying on pip for installing dpctl in the script, and reworks the script to maintain use of scikit-build

In the future, this will also simplify a transition to scikit-build-core

The PR also introduces options

--clean and --skip-editable to build_locally driver
--clean and --skip-pytest to gen_coverage driver
--clean to gen_docs driver

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to an issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
Have you checked performance impact of proposed changes?
Have you added documentation for your changes, if necessary?
Have you added your changes to the changelog?
If this PR is a work in progress, are you opening the PR as a draft?

github-actions · 2025-10-15T01:06:39Z

View rendered docs @ https://intelpython.github.io/dpctl/pulls/2172/index.html

github-actions · 2025-10-15T01:11:01Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_29 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-15T01:14:34Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_29 ran successfully.
Passed: 1116
Failed: 5
Skipped: 119

coveralls · 2025-10-15T01:15:53Z

coverage: 86.277%. remained the same
when pulling 5b395e7 on do-not-use-setup-py-develop
into 091beaa on master.

github-actions · 2025-10-15T05:30:47Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_30 ran successfully.
Passed: 1114
Failed: 7
Skipped: 119

to be reused in gen_docs and gen_coverage

github-actions · 2025-10-15T05:58:07Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_30 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-15T06:22:57Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_30 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

scripts/_build_helper.py

scripts/build_locally.py

antonwolfy · 2025-10-15T10:10:51Z

scripts/build_locally.py

+    p.add_argument(
        "--debug",
-        default="Release",
+        dest="build_type",


The debug build produces a lot of the warning which probably has to be resolved in the follow-up PR.

a number of them are
icpx: warning: argument unused during compilation: '-Xsycl-target-frontend=spir64 -g0'
which will need some looking at

the remainder are
warning: comparison of integers of different signs: 'py::ssize_t' (aka 'long') and 'std::size_t'
from
assert(dst.get_size() == iter_nelems)
in sorting functions

should be a simple enough fix

github-actions · 2025-10-15T16:14:26Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_32 ran successfully.
Passed: 1116
Failed: 5
Skipped: 119

github-actions · 2025-10-15T16:21:18Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_31 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-15T21:39:04Z

Array API standard conformance tests for dpctl=0.21.0=py310h93fe807_32 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

Update `gen_coverage` and `gen_docs` to align with `build_locally`

github-actions · 2025-10-28T23:19:35Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_33 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-28T23:26:34Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_34 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-28T23:43:44Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_34 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

github-actions · 2025-10-29T00:38:14Z

Array API standard conformance tests for dpctl=0.22.0dev0=py310h93fe807_35 ran successfully.
Passed: 1115
Failed: 6
Skipped: 119

antonwolfy · 2025-10-30T15:31:24Z

.github/workflows/conda-package.yml

            pushd $d
            conda activate --stack ${{ env.BUILD_ENV_NAME }}
-            CC=icx CXX=icpx python setup.py develop -G Ninja  || exit 1
+            CC=icx CXX=icpx python setup.py build_ext --inplace -G Ninja  || exit 1


CONTRIBUTING.md needs to be updated also

and docs/_legacy/docfiles/user_guides/QuickStart.rst and docs/doc_sources/contributor_guides/building.rst

some README.md in the examples folder as well

.github/workflows/generate-docs.yml

.github/workflows/os-llvm-sycl-build.yml

antonwolfy · 2025-10-30T15:49:51Z

scripts/build_locally.py

 import sys

+# add scripts dir to Python path so we can import _build_helper
+sys.path.insert(0, os.path.abspath("scripts"))


I wonder for what use case we need that? Simple python scripts/build_locally.py works fine without that.

It didn't work for me locally without this, essentially importing from _build_helper wouldn't work without this or an __init__.py

scripts/gen_coverage.py

antonwolfy · 2025-10-30T16:01:48Z

scripts/gen_coverage.py

+        "--bin-llvm",
+        help="Path to folder where llvm-cov/llvm-profdata can be found",
+    )
+    p.add_argument("--skip-pytest", dest="run_pytest", action="store_false")


Missing description of the argument

antonwolfy · 2025-10-30T16:39:33Z

scripts/gen_docs.py

+    env = os.environ.copy()
+
+    if "CMAKE_ARGS" in env and env["CMAKE_ARGS"].strip():
+        warn("Ignoring pre-existing CMAKE_ARGS in environment", "gen_docs")
+        del env["CMAKE_ARGS"]
+
+    env["CMAKE_ARGS"] = cmake_args
+
+    print(f"[gen_docs] Using CMake args:\n {env['CMAKE_ARGS']}")


Would it make sense to move that to a helper function, looks quite the same use across the files?

I plan to remove this, I was mistakenly under the impression that we needed to pass via CMAKE_ARGS when using python setup.py build_ext --inplace, but I can get it working without it

antonwolfy · 2025-10-30T17:06:41Z

scripts/build_locally.py

+        )
+
+    # Level Zero state (on unless explicitly disabled)
+    if args.no_level_zero:


I wonder, if --no-level-zero is deprecated, should we base the logic mostly on --target-level-zero option then?
Should we consider level_zero_enabled = False if --target-level-zero is not passed?

antonwolfy · 2025-10-30T17:08:15Z

scripts/build_locally.py

+        dest="no_level_zero",
+        action="store_true",
+        default=False,
+        help="Disable Level Zero backend (deprecated: use --target-level-zero "


--target-level-zero OFF does not work

antonwolfy · 2025-10-30T17:10:11Z

scripts/build_locally.py

+    if args.no_level_zero and args.target_level_zero:
+        err(
+            "Cannot combine --no-level-zero and --target-level-zero",
+            "build_locally",
+        )
+
+    # Level Zero state (on unless explicitly disabled)
+    if args.no_level_zero:
+        level_zero_enabled = False
+    elif args.target_level_zero:
+        level_zero_enabled = True


This is the same in all three scripts and can be moved to the helper as a function.

update build_locally script to avoid python setup.py develop call

810afd7

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from 6d87c72 to 124f5f7 Compare October 15, 2025 00:34

further update build_locally script

32c7b9f

ndgrigorian force-pushed the do-not-use-setup-py-develop branch 2 times, most recently from fd45fec to 13409b7 Compare October 15, 2025 05:18

refactor common build functionality out into separate file

32f1509

to be reused in gen_docs and gen_coverage

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from 13409b7 to 32f1509 Compare October 15, 2025 05:42

ndgrigorian changed the title ~~Do not use python setup.py develop in the project~~ Remove use of python setup.py develop/install in the project Oct 15, 2025

antonwolfy reviewed Oct 15, 2025

View reviewed changes

remove --build-dir option and fix --clean option

989df1b

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from 00161f1 to 989df1b Compare October 15, 2025 15:40

ndgrigorian requested a review from antonwolfy October 15, 2025 15:46

do not return compiler root unnecessarily from resolve_compilers

0648317

ndgrigorian added 7 commits October 20, 2025 17:19

update gen_coverage script to align with build_locally

a4d90be

pass explicit CMake variables for coverage tool paths

7b53623

replicate old script behavior when calling setup.py

72ca5a0

attempt setting cmake_args for LLVMCov_EXE

80af656

resolve bin_llvm from default compiler layout when not provided

527de4f

use bin_llvm when running llvm-cov and llvm-profdata

8973006

keep find_objects defined within main

50394b1

ndgrigorian and others added 5 commits October 28, 2025 10:55

generalize err and warn utilities for different build scripts

f327741

use common resolve_compilers utility

6c33bf0

update gen_docs script

35f91ea

Merge pull request #2174 from IntelPython/update-gen-coverage-script

74947d6

Update `gen_coverage` and `gen_docs` to align with `build_locally`

try using pip install -e in place of setup.py develop in CI

a113d01

ndgrigorian force-pushed the do-not-use-setup-py-develop branch from acb4584 to a113d01 Compare October 28, 2025 23:03

ndgrigorian marked this pull request as ready for review October 28, 2025 23:54

ndgrigorian requested a review from vlad-perevezentsev as a code owner October 28, 2025 23:54

use python setup.py build_ext in Cython extension building

5b395e7

antonwolfy reviewed Oct 30, 2025

View reviewed changes

Remove use of python setup.py develop/install in the project #2172

Are you sure you want to change the base?

Remove use of python setup.py develop/install in the project #2172

Conversation

ndgrigorian commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

coveralls commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 15, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

github-actions bot commented Oct 28, 2025

Uh oh!

github-actions bot commented Oct 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Remove use of `python setup.py develop/install` in the project #2172

Remove use of `python setup.py develop/install` in the project #2172

ndgrigorian commented Oct 15, 2025 •

edited

Loading

coveralls commented Oct 15, 2025 •

edited

Loading