Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4 and SQLite >= 3.37. #19107

charettes · 2025-01-27T20:15:38Z

Per discussion

Trac ticket number

ticket-36148

Branch description

I can't get my hands on Oracle <= 23.3 but I know that

CI which is running 19 doesn't support it
It seems to be missing on Oracle 23.2
It's working on Oracle 23.4
ORACLE_VERSION=23.4.0.0 PYTHON_VERSION=3.12 docker compose run --rm oracle --parallel 1 --keepdb composite_pk passes
ORACLE_VERSION=23.5.0.0 PYTHON_VERSION=3.12 docker compose run --rm oracle --parallel 1 --keepdb composite_pk passes

charettes · 2025-01-27T20:20:51Z

django/db/backends/oracle/features.py

This can be handy in bulk_batch_size as if this is supported we can likely avoid considering one placeholder per composite field see https://github.com/django/django/pull/19088/files#r1931008541.

charettes · 2025-01-28T20:20:43Z

Discovered that SQLite does support native tuple containment checks by digging a bit

felixxm · 2025-02-02T20:42:49Z

Per discussion

Trac ticket number

ticket-36148

Branch description

I can't get my hands on Oracle <= 23.3 but I know that
* CI which is running 19 doesn't support it

* [It seems to be missing on Oracle 23.2](https://dbfiddle.uk/Z1p0AxvV)

* [It's working on Oracle 23.4](https://onecompiler.com/oracle/437c9m79g)

Oracle 23c has been rebranded to Oracle 23ai (starting with version 23.4), which would explain many new features in the "minor" release.

charettes · 2025-02-02T21:42:15Z

@felixxm I just spent some more time going through the release notes again and I couldn't any mention of it.

It might have been a feature that flew the radar though given that support the VALUES constructor was added and it's effectively a way to define a sequence of rows.

That would explain why (c0, ..., cn) IN (SELECT s0, ..., sn FROM ...) was supported on Oracle 19 but not (c0, ..., cn) = (L0, ..., LN) for literal values.

Looking how we circumvented the lack of VALUES support for Oracle on inserts I suspect the same could work for literal values instead!

charettes · 2025-02-02T21:50:20Z

There we go it seems that we can work around the TupleExact.as_oracle on 19 by using (SELECT L0, ..., L5 FROM DUAL) instead of (L0, ..., L5).

It appears that < and > are not implemented though.

felixxm · 2025-02-06T21:08:38Z

Looking how we circumvented the lack of VALUES support for Oracle on inserts I suspect the same could work for literal values instead!

Unfortunately we had to revert 175b049 (see 5424151 and ticket-35655).

django/db/models/fields/tuple_lookups.py

charettes · 2025-02-06T21:13:37Z

Unfortunately we had to revert 175b049 (see 5424151 and ticket-35655).

The issue might only be for INSERT though an not for (c0, ..., cn) IN (SELECT v0, ..., vn FROM DUAL). Not sure it's worth doing given it allows us to exercise the fallback code anyway.

django/db/models/fields/tuple_lookups.py

django/db/backends/sqlite3/features.py

This should allow third-party backends to define Tuple.as_vendor() overrides that are taken into consideration which calling as_sql() directly prevents.

…port. This should allow backends more easily opt-in or out of native support and rely on the fallback if unavailable.

…+ and Oracle 23.4+. VALUES must be explicitly specified when declaring a sequence of tuples on SQLite < 3.37 but it's not required on >= 3.37. See sqlite/sqlite@9289f51 which addressed the last remaining issue with IN.

felixxm

Thanks 👍

felixxm · 2025-02-09T16:47:11Z

Merged in c326cfe, a0a765d, and 4a3ad9e.

github-actions bot added the no ticket Based on PR title, no linked Trac ticket label Jan 27, 2025

charettes commented Jan 27, 2025

View reviewed changes

charettes mentioned this pull request Jan 27, 2025

Fixed #36118 -- Accounted for multiple primary keys in bulk_update max_batch_size. #19088

Merged

charettes force-pushed the tuple-lookups_as_fallback branch 3 times, most recently from bb6a5aa to 6d0e2ce Compare January 28, 2025 01:05

charettes changed the title ~~Enabled native tuple comparison lookups on Oracle >= 23.4.~~ Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4. Jan 28, 2025

github-actions bot removed the no ticket Based on PR title, no linked Trac ticket label Jan 28, 2025

charettes marked this pull request as ready for review January 28, 2025 01:07

charettes force-pushed the tuple-lookups_as_fallback branch from 6d0e2ce to 9952fc4 Compare January 28, 2025 20:19

felixxm self-requested a review January 29, 2025 15:56

felixxm reviewed Feb 6, 2025

View reviewed changes

django/db/models/fields/tuple_lookups.py Outdated Show resolved Hide resolved

charettes force-pushed the tuple-lookups_as_fallback branch from 9952fc4 to b6898ff Compare February 8, 2025 14:53

charettes commented Feb 8, 2025

View reviewed changes

django/db/models/fields/tuple_lookups.py Outdated Show resolved Hide resolved

charettes force-pushed the tuple-lookups_as_fallback branch from b6898ff to 3016df2 Compare February 8, 2025 16:13

charettes commented Feb 8, 2025

View reviewed changes

django/db/models/fields/tuple_lookups.py Outdated Show resolved Hide resolved

charettes changed the title ~~Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4.~~ Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4 and SQLite > 3.37. Feb 8, 2025

charettes changed the title ~~Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4 and SQLite > 3.37.~~ Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4 and SQLite >= 3.37. Feb 8, 2025

charettes force-pushed the tuple-lookups_as_fallback branch from 3016df2 to 47ab068 Compare February 9, 2025 02:09

felixxm reviewed Feb 9, 2025

View reviewed changes

django/db/backends/sqlite3/features.py Outdated Show resolved Hide resolved

charettes added 3 commits February 9, 2025 15:15

Refs django#373 -- Delegated Tuple compilation to compiler.

c326cfe

This should allow third-party backends to define Tuple.as_vendor() overrides that are taken into consideration which calling as_sql() directly prevents.

Refs #36148 -- Relied on a feature switch to define tuple lookups sup…

a0a765d

…port. This should allow backends more easily opt-in or out of native support and rely on the fallback if unavailable.

felixxm force-pushed the tuple-lookups_as_fallback branch from 1ae16d3 to bfc7364 Compare February 9, 2025 14:15

felixxm approved these changes Feb 9, 2025

View reviewed changes

felixxm closed this Feb 9, 2025

charettes deleted the tuple-lookups_as_fallback branch February 9, 2025 16:47

Uh oh!

Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4 and SQLite >= 3.37. #19107

Fixed #36148 -- Enabled native tuple comparison lookups on Oracle >= 23.4 and SQLite >= 3.37. #19107

Uh oh!

Conversation

charettes commented Jan 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Trac ticket number

Branch description

Uh oh!

charettes Jan 27, 2025

Choose a reason for hiding this comment

Uh oh!

charettes commented Jan 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felixxm commented Feb 2, 2025

Trac ticket number

Branch description

Uh oh!

charettes commented Feb 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charettes commented Feb 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

felixxm commented Feb 6, 2025

Uh oh!

Uh oh!

charettes commented Feb 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

felixxm left a comment

Choose a reason for hiding this comment

Uh oh!

felixxm commented Feb 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

charettes commented Jan 27, 2025 •

edited

Loading

charettes commented Jan 28, 2025 •

edited

Loading

charettes commented Feb 2, 2025 •

edited

Loading

charettes commented Feb 2, 2025 •

edited

Loading