mirror of
https://github.com/django/django.git
synced 2025-10-31 09:41:08 +00:00
258 lines
10 KiB
Plaintext
258 lines
10 KiB
Plaintext
==============
|
|
Custom lookups
|
|
==============
|
|
|
|
.. module:: django.db.models.lookups
|
|
:synopsis: Custom lookups
|
|
|
|
.. currentmodule:: django.db.models
|
|
|
|
By default Django offers a wide variety of different lookups for filtering
|
|
(for example, `exact` and `icontains`). This documentation explains how to
|
|
write custom lookups and how to alter the working of existing lookups. In
|
|
addition how to transform field values is explained. fFor example how to
|
|
extract the year from a DateField. By writing a custom `YearExtract`
|
|
transformer it is possible to filter on the transformed value, for example::
|
|
|
|
Author.objects.filter(birthdate__year__lte=1981)
|
|
|
|
Currently transformers are only available in filtering. So, it is not possible
|
|
to use it in other parts of the ORM, for example this will not work::
|
|
|
|
Author.objects.values_list('birthdate__year')
|
|
|
|
A simple Lookup example
|
|
~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Lets start with a simple custom lookup. We will write a custom lookup `ne`
|
|
which works opposite to `exact`. A `Author.objects.filter(name__ne='Jack')`
|
|
will translate to::
|
|
|
|
"author"."name" <> 'Jack'
|
|
|
|
A custom lookup will need an implementation and Django needs to be told
|
|
the existence of the lookup. The implementation for this lookup will be
|
|
simple to write::
|
|
|
|
from django.db.models import Lookup
|
|
|
|
class NotEqual(Lookup):
|
|
lookup_name = 'ne'
|
|
|
|
def as_sql(self, qn, connection):
|
|
lhs, lhs_params = self.process_lhs(qn, connection)
|
|
rhs, rhs_params = self.process_rhs(qn, connection)
|
|
params = lhs_params + rhs_params
|
|
return '%s <> %s' % (lhs, rhs), params
|
|
|
|
To register the `NotEqual` lookup we will just need to call register_lookup
|
|
on the field class we want the lookup to be available::
|
|
|
|
from django.db.models.fields import Field
|
|
Field.register_lookup(NotEqual)
|
|
|
|
Now Field and all its subclasses have a NotEqual lookup.
|
|
|
|
The first notable thing about `NotEqual` is the lookup_name. This name must
|
|
be supplied, and it is used by Django in the register_lookup() call so that
|
|
Django knows to associate `ne` to the NotEqual implementation.
|
|
`
|
|
An Lookup works against two values, lhs and rhs. The abbreviations stand for
|
|
left-hand side and right-hand side. The lhs is usually a field reference,
|
|
but it can be anything implementing the query expression API. The
|
|
rhs is the value given by the user. In the example `name__ne=Jack`, the
|
|
lhs is reference to Author's name field and Jack is the value.
|
|
|
|
The lhs and rhs are turned into values that are possible to use in SQL.
|
|
In the example above lhs is turned into "author"."name", [], and rhs is
|
|
turned into "%s", ['Jack']. The lhs is just raw string without parameters
|
|
but the rhs is turned into a query parameter 'Jack'.
|
|
|
|
Finally we combine the lhs and rhs by adding ` <> ` in between of them,
|
|
and supply all the parameters for the query.
|
|
|
|
A Lookup needs to implement a limited part of query expression API. See
|
|
the query expression API for details.
|
|
|
|
A simple transformer example
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
We will next write a simple transformer. The transformer will be called
|
|
`YearExtract`. It can be used to extract the year part from `DateField`.
|
|
|
|
Lets start by writing the implementation::
|
|
|
|
from django.db.models import Extract
|
|
|
|
class YearExtract(Extract):
|
|
lookup_name = 'year'
|
|
output_type = IntegerField()
|
|
|
|
def as_sql(self, qn, connection):
|
|
lhs, params = qn.compile(self.lhs)
|
|
return "EXTRACT(YEAR FROM %s)" % lhs, params
|
|
|
|
Next, lets register it for `DateField`::
|
|
|
|
from django.db.models import DateField
|
|
DateField.register_lookup(YearExtract)
|
|
|
|
Now any DateField in your project will have `year` transformer. For example
|
|
the following query::
|
|
|
|
Author.objects.filter(birthdate__year__lte=1981)
|
|
|
|
would translate to the following query on PostgreSQL::
|
|
|
|
SELECT ...
|
|
FROM "author"
|
|
WHERE EXTRACT(YEAR FROM "author"."birthdate") <= 1981
|
|
|
|
An YearExtract class works only against self.lhs. Usually the lhs is
|
|
transformed in some way. Further lookups and extracts work against the
|
|
transformed value.
|
|
|
|
Note the definition of output_type in the `YearExtract`. The output_type is
|
|
a field instance. It informs Django that the Extract class transformed the
|
|
type of the value to an int. This is currently used only to check which
|
|
lookups the extract has.
|
|
|
|
The used SQL in this example works on most databases. Check you database
|
|
vendor's documentation to see if EXTRACT(year from date) is supported.
|
|
|
|
Writing an efficient year__exact lookup
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
When using the above written `year` lookup, the SQL produced will not use
|
|
indexes efficiently. We will fix that by writing a custom `exact` lookup
|
|
for YearExtract. For example if the user filters on
|
|
`birthdate__year__exact=1981`, then we want to produce the following SQL::
|
|
|
|
birthdate >= to_date('1981-01-01') AND birthdate <= to_date('1981-12-31')
|
|
|
|
The implementation is::
|
|
|
|
from django.db.models import Lookup
|
|
|
|
class YearExact(Lookup):
|
|
lookup_name = 'exact'
|
|
|
|
def as_sql(self, qn, connection):
|
|
lhs, lhs_params = qn.compile(self.lhs.lhs)
|
|
rhs, rhs_params = self.process_rhs(qn, connection)
|
|
params = lhs_params + rhs_params + lhs_params + rhs_params
|
|
return '%s >= to_date(%s || '-01-01') AND %s <= to_date(%s || '-12-31') % (lhs, rhs, lhs, rhs), params
|
|
|
|
YearExtract.register_lookup(YearExact)
|
|
|
|
There are a couple of notable things going on. First, `YearExact` isn't
|
|
calling process_lhs(). Instead it skips and compiles directly the lhs used by
|
|
self.lhs. The reason this is done is to skip `YearExtract` from adding the
|
|
EXTRACT clause to the query. Referring directly to self.lhs.lhs is safe as
|
|
`YearExact` can be accessed only from `year__exact` lookup, that is the lhs
|
|
is always `YearExtract`.
|
|
|
|
Next, as both the lhs and rhs are used multiple times in the query the params
|
|
need to contain lhs_params and rhs_params multiple times.
|
|
|
|
The final query does string manipulation directly in the database. The reason
|
|
for doing this is that if the self.rhs is something else than a plain integer
|
|
value (for exampel a `F()` reference) we can't do the transformations in
|
|
Python.
|
|
|
|
Writing alternative implemenatations for existing lookups
|
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
Sometimes different database vendors require different SQL for the same
|
|
operation. For this example we will rewrite a custom implementation for
|
|
MySQL for the NotEqual operator. Instead of `<>` we will be using `!=`
|
|
operator.
|
|
|
|
There are two ways to do this. The first is to write a subclass with a
|
|
as_mysql() method and registering the subclass over the original class::
|
|
|
|
class MySQLNotEqual(NotEqual):
|
|
def as_mysql(self, qn, connection):
|
|
lhs, lhs_params = self.process_lhs(qn, connection)
|
|
rhs, rhs_params = self.process_rhs(qn, connection)
|
|
params = lhs_params + rhs_params
|
|
return '%s != %s' % (lhs, rhs), params
|
|
Field.register_lookup(MySQLNotExact)
|
|
|
|
The alternate is to monkey-patch the existing class in place::
|
|
def as_mysql(self, qn, connection):
|
|
lhs, lhs_params = self.process_lhs(qn, connection)
|
|
rhs, rhs_params = self.process_rhs(qn, connection)
|
|
params = lhs_params + rhs_params
|
|
return '%s != %s' % (lhs, rhs), params
|
|
NotEqual.as_mysql = as_mysql
|
|
|
|
The subclass way allows one to override methods of the lookup if needed. The
|
|
monkey-patch way allows writing different implementations for the same class
|
|
in different locations of the project.
|
|
|
|
The way Django knows to call as_mysql() instead of as_sql() is as follows.
|
|
When qn.compile(notequal_instance) is called, Django first checks if there
|
|
is a method named 'as_%s' % connection.vendor. If that method doesn't exist,
|
|
the as_sql() will be called.
|
|
|
|
The vendor names for Django's in-built backends are 'sqlite', 'postgresql',
|
|
'oracle' and 'mysql'.
|
|
|
|
The Lookup API
|
|
~~~~~~~~~~~~~~
|
|
|
|
An lookup has attributes lhs and rhs. The lhs is something implementing the
|
|
query expression API and the rhs is either a plain value, or something that
|
|
needs to be compiled into SQL. Examples of SQL-compiled values include `F()`
|
|
references and usage of `QuerySets` as value.
|
|
|
|
A lookup needs to define lookup_name as a class level attribute. This is used
|
|
when registering lookups.
|
|
|
|
A lookup has three public methods. The as_sql(qn, connection) method needs
|
|
to produce a query string and parameters used by the query string. The qn has
|
|
a method compile() which can be used to compile self.lhs. However usually it
|
|
is better to call self.process_lhs(qn, connection) instead, which returns
|
|
query string and parameters for the lhs. Similary process_rhs(qn, connection)
|
|
returns query string and parameters for the rhs.
|
|
|
|
The Query Expression API
|
|
~~~~~~~~~~~~~~~~~~~~~~~~
|
|
|
|
A lookup can assume that the lhs responds to the query expression API.
|
|
Currently direct field references, aggregates and `Extract` instances respond
|
|
to this API.
|
|
|
|
.. method:: as_sql(qn, connection)
|
|
|
|
Responsible for producing the query string and parameters for the expression.
|
|
The qn has a compile() method that can be used to compile other expressions.
|
|
The connection is the connection used to execute the query. The
|
|
connection.vendor attribute can be used to return different query strings
|
|
for different backends.
|
|
|
|
Calling expression.as_sql() directly is usually an error - instead
|
|
qn.compile(expression) should be used. The qn.compile() method will take
|
|
care of calling vendor-specific methods of the expression.
|
|
|
|
.. method:: as_vendorname(qn, connection)
|
|
|
|
Works like as_sql() method. When an expression is compiled by qn.compile()
|
|
Django will first try to call as_vendorname(), where vendorname is the vendor
|
|
name of the backend used for executing the query. The vendorname is one of
|
|
'postgresql', 'oracle', 'sqlite' or 'mysql' for Django's inbuilt backends.
|
|
|
|
.. method:: get_lookup(lookup_name)::
|
|
|
|
The get_lookup() method is used to fetch lookups. By default the lookup
|
|
is fetched from the expression's output type, but it is possible to override
|
|
this method to alter that behaviour.
|
|
|
|
.. attribute:: output_type
|
|
|
|
The output_type attribute is used by the get_lookup() method to check for
|
|
lookups. The output_type should be a field instance.
|
|
|
|
Note that this documentation lists only the public methods of the API.
|