====================== Database API reference ====================== Once you've created your `data models`_, you'll need to retrieve data from the database. This document explains the database abstraction API derived from the models, and how to create, retrieve and update objects. .. _`data models`: http://www.djangoproject.com/documentation/model_api/ Throughout this reference, we'll refer to the following Poll application:: class Poll(models.Model): slug = models.SlugField(unique_for_month='pub_date') question = models.CharField(maxlength=255) pub_date = models.DateTimeField() expire_date = models.DateTimeField() def __repr__(self): return self.question class Meta: get_latest_by = 'pub_date' class Choice(models.Model): poll = models.ForeignKey(Poll, edit_inline=meta.TABULAR, num_in_admin=10, min_num_in_admin=5) choice = models.CharField(maxlength=255, core=True) votes = models.IntegerField(editable=False, default=0) def __repr__(self): return self.choice and the following Django sample session:: >>> from datetime import datetime >>> p1 = Poll(slug='whatsup', question="What's up?", ... pub_date=datetime(2005, 2, 20), expire_date=datetime(2005, 4, 20)) >>> p1.save() >>> p2 = Poll(slug='name', question="What's your name?", ... pub_date=datetime(2005, 3, 20), expire_date=datetime(2005, 3, 25)) >>> p2.save() >>> Poll.objects.all() [What's up?, What's your name?] How Queries Work ================ Querying in Django is based upon the construction and evaluation of Query Sets. A Query Set is a database-independent representation of a query. It can be thought of as a representation of a group of objects that meet a given set of criteria. However, the members of the set are not determined until the Query Set is formally evaluated. To compose a Query using Django, you obtain an initial a Query Set. This Query Set can then be refined using a range of operations. When you have a Query Set that meets your needs, it can be evaluated (using iterators, slicing, or one of a range of other techniques), yielding an object or list of objects that meet the specifications of the Query Set. Obtaining a Query Set ===================== Query Sets are obtained using the Manager object on a model. Every model has at least one Manager; by default, the Manager is called ``objects``. See the `Managers`_ section of the Model API for more details on the role and construction of Managers. .. _Managers: http://www.djangoproject.com/documentation/model_api/#managers The manager has a special factory method for creating Query Sets:: queryset = Poll.objects.all() This creates a new Query Set that matches all the objects of the given class. As a convenient shortcut, all of these Query Set construction methods can be accessed from the Manager object itself. The following two queries are identical:: Poll.objects.all().filter(question__startswith="What") Poll.objects.filter(question__startswith="What") Query Set Refinement ==================== The default Query Set returned by the Manager contains all objects of the Model type. In order to be useful, Any Query Set can be refined by calling one of the following methods: filter(\**kwargs) Returns a new Query Set containing objects that match the given lookup parameters. exclude(\**kwargs) Return a new Query Set containing objects that do not match the given lookup parameters. Lookup parameters should be in the format described in "Field lookups" below. Query Set refinements can be chained together:: Poll.objects.filter(question__startswith="What").exclude().filter(...) Query Sets can also be stored and reused:: q1 = Poll.objects.filter() q2 = q1.exclude() q3 = q1.filter() Field lookups ============= Basic field lookups take the form ``field__lookuptype`` (that's a double-underscore). For example:: Poll.objects.filter(pub_date__lte=datetime.datetime.now()) translates (roughly) into the following SQL:: SELECT * FROM polls_poll WHERE pub_date <= NOW(); .. admonition:: How this is possible Python has the ability to define functions that accept arbitrary name-value arguments whose names and values are evaluated at run time. For more information, see `Keyword Arguments`_ in the official Python tutorial. The DB API supports the following lookup types: =========== ============================================================== Type Description =========== ============================================================== exact Exact match: ``Poll.objects.get(id__exact=14)`` returns all polls with an ID of exactly 14. iexact Case-insensitive exact match: ``Poll.objects.filter(slug__iexact="foo")`` matches a slug of ``foo``, ``FOO``, ``fOo``, etc. contains Case-sensitive containment test: ``Poll.objects.filter(question__contains="spam")`` returns all polls that contain "spam" in the question. (PostgreSQL and MySQL only. SQLite doesn't support case-sensitive LIKE statements; ``contains`` will act like ``icontains`` for SQLite.) icontains Case-insensitive containment test. gt Greater than: ``Poll.objects.filter(id__gt=4)``. gte Greater than or equal to. lt Less than. lte Less than or equal to. in In a given list: ``Poll.objects.filter(id__in=[1, 3, 4])`` returns a list of polls whose IDs are either 1, 3 or 4. startswith Case-sensitive starts-with: ``Poll.objects.filter(question__startswith="Would")``. (PostgreSQL and MySQL only. SQLite doesn't support case-sensitive LIKE statements; ``startswith`` will act like ``istartswith`` for SQLite.) endswith Case-sensitive ends-with. (PostgreSQL and MySQL only.) istartswith Case-insensitive starts-with. iendswith Case-insensitive ends-with. range Range test: ``Poll.objects.filter(pub_date__range=(start_date, end_date))`` returns all polls with a pub_date between ``start_date`` and ``end_date`` (inclusive). year For date/datetime fields, exact year match: ``Poll.objects.count(pub_date__year=2005)``. month For date/datetime fields, exact month match. day For date/datetime fields, exact day match. isnull True/False; does is IF NULL/IF NOT NULL lookup: ``Poll.objects.filter(expire_date__isnull=True)``. =========== ============================================================== If no lookup type is provided, a type of ``exact`` is assumed. The following two statements are equivalent:: Poll.objects.get(id=14) Poll.objects.get(id__exact=14) Multiple lookups are also allowed. When separated by commans, the list of lookups will be "AND"ed together:: Poll.objects.filter( pub_date__year=2005, pub_date__month=1, question__startswith="Would", ) ...retrieves all polls published in January 2005 that have a question starting with "Would." For convenience, there's a ``pk`` lookup type, which translates into ``(primary_key)``. In the polls example, these two statements are equivalent:: Poll.objects.get(id__exact=3) Poll.objects.get(pk=3) ``pk`` lookups also work across joins. In the polls example, these two statements are equivalent:: Choice.objects.filter(poll__id=3) Choice.objects.filter(poll__pk=3) If you pass an invalid keyword argument, the function will raise ``TypeError``. .. _`Keyword Arguments`: http://docs.python.org/tut/node6.html#SECTION006720000000000000000 Query Set evaluation ==================== Once you have constructed a Query Set to meet your needs, it must be evaluated to return the objects that are contained in the set. This can be achieved in A Query Set is an iterable object:: queryset = Poll.objects.all() for p in queryset: print p Query Sets can also be sliced:: fifth_poll = queryset[4] all_polls_but_the_first_two = queryset[2:] If you really need to have a . :: querylist = list(Poll.objects.all()) However - be warned; if you use these approaches, Regardless of whether you iterate or slice the Query Set, upon first evaluation, the query will be executed on the database, and the results cached. Subsequent evaluations of the Query Set reuse the cached results. As an alternative to iteration and slicing, you can use one of the following functions. These functions do not populate or effect the cache: get(\**kwargs) -------------- Returns the object matching the given lookup parameters, which should be in the format described in _`Field lookups`. Raises a module-level ``DoesNotExist`` exception if an object wasn't found for the given parameters. Raises ``AssertionError`` if more than one object was found. count() ------- Returns an integer representing the number of objects in the database matching the Query Set. ``count()`` never raises exceptions. Depending on which database you're using (e.g. PostgreSQL vs. MySQL), this may return a long integer instead of a normal Python integer. in_bulk(id_list) ---------------- Takes a list of IDs and returns a dictionary mapping each ID to an instance of the object with the given ID. For example:: >>> Poll.objects.in_bulk([1]) {1: What's up?} >>> Poll.objects.in_bulk([1, 2]) {1: What's up?, 2: What's your name?} >>> Poll.objects.in_bulk([]) {} latest(field_name=None) ----------------------- Returns the latest object, according to the model's 'get_latest_by' Meta option, or using the field_name provided. For example:: >>> Poll.objects.latest() What's up? >>> Poll.objects.latest('expire_date') What's your name? OR lookups ========== By default, keyword argument queries are "AND"ed together. If you have more complex query requirements (for example, you need to include an ``OR`` statement in your query), you need to use ``Q`` objects. A ``Q`` object (``django.db.models.Q``) is an object used to encapsulate a collection of keyword arguments. These keyword arguments are specified in the same way as keyword arguments to the basic lookup functions like get() and filter(). For example:: Q(question__startswith='What') is a ``Q`` object encapsulating a single ``LIKE`` query. ``Q`` objects can be combined using the ``&`` and ``|`` operators. When an operator is used on two ``Q`` objects, it yields a new ``Q`` object. For example the statement:: Q(question__startswith='Who') | Q(question__startswith='What') ... yields a single ``Q`` object that represents the "OR" of two "question__startswith" queries, equivalent to the SQL WHERE clause:: ... WHERE question LIKE 'Who%' OR question LIKE 'What%' You can compose statements of arbitrary complexity by combining ``Q`` objects with the ``&`` and ``|`` operators. Parenthetical grouping can also be used. One or more ``Q`` objects can then provided as arguments to the lookup functions. If multiple ``Q`` object arguments are provided to a lookup function, they will be "AND"ed together. For example:: Poll.objects.get( Q(question__startswith='Who'), Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6)) ) ... roughly translates into the SQL:: SELECT * from polls WHERE question LIKE 'Who%' AND (pub_date = '2005-05-02' OR pub_date = '2005-05-06') If necessary, lookup functions can mix the use of ``Q`` objects and keyword arguments. All arguments provided to a lookup function (be they keyword argument or ``Q`` object) are "AND"ed together. However, if a ``Q`` object is provided, it must precede the definition of any keyword arguments. For example:: Poll.objects.get( Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6)), question__startswith='Who') ... would be a valid query, equivalent to the previous example; but:: # INVALID QUERY Poll.objects.get( question__startswith='Who', Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6))) ... would not be valid. A ``Q`` objects can also be provided to the ``complex`` keyword argument. For example:: Poll.objects.get( complex=Q(question__startswith='Who') & (Q(pub_date=date(2005, 5, 2)) | Q(pub_date=date(2005, 5, 6)) ) ) See the `OR lookups examples page`_ for more examples. .. _OR lookups examples page: http://www.djangoproject.com/documentation/models/or_lookups/ Relationships (joins) ===================== Joins may implicitly be performed by following relationships: ``Choice.objects.filter(poll__slug="eggs")`` fetches a list of ``Choice`` objects where the associated ``Poll`` has a slug of ``eggs``. Multiple levels of joins are allowed. Given an instance of an object, related objects can be looked-up directly using convenience functions. For example, if ``p`` is a ``Poll`` instance, ``p.choice_set.all()`` will return a list of all associated choices. Astute readers will note that this is the same as ``Choice.objects.filter(poll__id=p.id)``, except clearer. Each type of relationship creates a set of methods on each object in the relationship. These methods are created in both directions, so objects that are "related-to" need not explicitly define reverse relationships; that happens automatically. One-to-one relations -------------------- Each object in a one-to-one relationship will have a ``get_relatedobjectname()`` method. For example:: class Place(models.Model): # ... class Restaurant(models.Model): # ... the_place = models.OneToOneField(Place) In the above example, each ``Place`` will have a ``get_restaurant()`` method, and each ``Restaurant`` will have a ``get_the_place()`` method. Many-to-one relations --------------------- In each many-to-one relationship, the related object will have a ``get_relatedobject()`` method, and the related-to object will have ``get_relatedobject()``, ``get_relatedobject_list()``, and ``get_relatedobject_count()`` methods (the same as the module-level ``get_object()``, ``filter()``, and ``get_count()`` methods). In the poll example above, here are the available choice methods on a ``Poll`` object ``p``:: p.get_choice() p.get_choice_list() p.get_choice_count() And a ``Choice`` object ``c`` has the following method:: c.get_poll() Many-to-many relations ---------------------- Many-to-many relations result in the same set of methods as `Many-to-one relations`_, except that the ``get_relatedobject_list()`` function on the related object will return a list of instances instead of a single instance. So, if the relationship between ``Poll`` and ``Choice`` was many-to-many, ``choice.get_poll_list()`` would return a list. Specialist Query Sets ===================== In addition to ``filter`` and ``exclude()``, Django provides a range of Query Set refinement methods that modify the types of results returned by the Query Set, or modify the way the SQL query is executed on the database. order_by(\*fields) ------------------ The results returned by a Query Set are automatically ordered by the ordering tuple given by the ``ordering`` meta key in the model. However, ordering may be explicitly provided by using the ``order_by`` method:: Poll.objects.filter(pub_date__year=2005, pub_date__month=1).order_by('-pub_date', 'question') The result set above will be ordered by ``pub_date`` descending, then by ``question`` ascending. The negative sign in front of "-pub_date" indicates descending order. Ascending order is implied. To order randomly, use "?", like so:: Poll.objects.order_by=('?') To order by a field in a different table, add the other table's name and a dot, like so:: Choice.objects.order_by=('Poll.pub_date', 'choice') There's no way to specify whether ordering should be case sensitive. With respect to case-sensitivity, Django will order results however your database backend normally orders them. distinct() ---------- By default, a Query Set will not eliminate duplicate rows. This will not happen during simple queries; however, if your query spans relations, or you are using a Values Query Set with a ``fields`` clause, it is possible to get duplicated results when a Query Set is evaluated. ``distinct()`` returns a new Query Set that eliminates duplicate rows from the results returned by the Query Set. This is equivalent to a ``SELECT DISTINCT`` SQL clause. values(\*fields) ---------------- Returns a Values Query Set - a Query Set that evaluates to a list of dictionaries instead of model-instance objects. Each dictionary in the list will represent an object matching the query, with the keys matching the attribute names of the object. It accepts an optional parameter, ``fields``, which should be a list or tuple of field names. If you don't specify ``fields``, each dictionary in the list returned by ``get_values()`` will have a key and value for each field in the database table. If you specify ``fields``, each dictionary will have only the field keys/values for the fields you specify. For example:: >>> Poll.objects.values() [{'id': 1, 'slug': 'whatsup', 'question': "What's up?", 'pub_date': datetime.datetime(2005, 2, 20), 'expire_date': datetime.datetime(2005, 3, 20)}, {'id': 2, 'slug': 'name', 'question': "What's your name?", 'pub_date': datetime.datetime(2005, 3, 20), 'expire_date': datetime.datetime(2005, 4, 20)}] >>> Poll.objects.values('id', 'slug') [{'id': 1, 'slug': 'whatsup'}, {'id': 2, 'slug': 'name'}] A Values Query Set is useful when you know you're only going to need values from a small number of the available fields and you won't need the functionality of a model instance object. It's more efficient to select only the fields you need to use. dates(field, kind, order='ASC') ------------------------------- Returns a Date Query Set - a Query Set that evaluates to a list of ``datetime.datetime`` objects representing all available dates of a particular kind within the contents of the Query Set. ``field`` should be the name of a ``DateField`` or ``DateTimeField`` of your model. ``kind`` should be either ``"year"``, ``"month"`` or ``"day"``. Each ``datetime.datetime`` object in the result list is "truncated" to the given ``type``. * ``"year"`` returns a list of all distinct year values for the field. * ``"month"`` returns a list of all distinct year/month values for the field. * ``"day"`` returns a list of all distinct year/month/day values for the field. ``order``, which defaults to ``'ASC'``, should be either ``"ASC"`` or ``"DESC"``. This specifies how to order the results. For example:: >>> Poll.objects.dates('pub_date', 'year') [datetime.datetime(2005, 1, 1)] >>> Poll.objects.dates('pub_date', 'month') [datetime.datetime(2005, 2, 1), datetime.datetime(2005, 3, 1)] >>> Poll.objects.dates('pub_date', 'day') [datetime.datetime(2005, 2, 20), datetime.datetime(2005, 3, 20)] >>> Poll.objects.dates('pub_date', 'day', order='DESC') [datetime.datetime(2005, 3, 20), datetime.datetime(2005, 2, 20)] >>> Poll.objects.filter(question__contains='name').dates('pub_date', 'day') [datetime.datetime(2005, 3, 20)] select_related() ---------------- Relations are the bread and butter of databases, so there's an option to "follow" all relationships and pre-fill them in a simple cache so that later calls to objects with a one-to-many relationship don't have to hit the database. Do this by passing ``select_related=True`` to a lookup. This results in (sometimes much) larger queries, but it means that later use of relationships is much faster. For example, using the Poll and Choice models from above, if you do the following:: c = Choice.objects.select_related().get(id=5) Then subsequent calls to ``c.poll`` won't hit the database. Note that ``select_related`` follows foreign keys as far as possible. If you have the following models:: class Poll(models.Model): # ... class Choice(models.Model): # ... poll = models.ForeignKey(Poll) class SingleVote(meta.Model): # ... choice = models.ForeignKey(Choice) then a call to ``SingleVotes.objects.select_related().get(id=4)`` will cache the related choice *and* the related poll:: >>> sv = SingleVotes.objects.select_related().get(id=4) >>> c = sv.choice # Doesn't hit the database. >>> p = c.poll # Doesn't hit the database. >>> sv = SingleVotes.objects.get(id=4) >>> c = sv.choice # Hits the database. >>> p = c.poll # Hits the database. extra(params, select, where, tables) ------------------------------------ Sometimes, the Django query syntax by itself isn't quite enough. To cater for these edge cases, Django provides the ``extra()`` Query Set modifier - a mechanism for injecting specific clauses into the SQL generated by a Query Set. Note that by definition these extra lookups may not be portable to different database engines (because you're explicitly writing SQL code) and should be avoided if possible.: ``params`` All the extra-SQL params described below may use standard Python string formatting codes to indicate parameters that the database engine will automatically quote. The ``params`` argument can contain any extra parameters to be substituted. ``select`` The ``select`` keyword allows you to select extra fields. This should be a dictionary mapping attribute names to a SQL clause to use to calculate that attribute. For example:: Poll.objects.extra( select={ 'choice_count': 'SELECT COUNT(*) FROM choices WHERE poll_id = polls.id' } ) Each of the resulting ``Poll`` objects will have an extra attribute, ``choice_count``, an integer count of associated ``Choice`` objects. Note that the parenthesis required by most database engines around sub-selects are not required in Django's ``select`` clauses. ``where`` / ``tables`` If you need to explicitly pass extra ``WHERE`` clauses -- perhaps to perform non-explicit joins -- use the ``where`` keyword. If you need to join other tables into your query, you can pass their names to ``tables``. ``where`` and ``tables`` both take a list of strings. All ``where`` parameters are "AND"ed to any other search criteria. For example:: Poll.objects.filter( question__startswith='Who').extra(where=['id IN (3, 4, 5, 20)']) ...translates (roughly) into the following SQL:: SELECT * FROM polls_polls WHERE question LIKE 'Who%' AND id IN (3, 4, 5, 20); Changing objects ================ Once you've retrieved an object from the database using any of the above options, changing it is extremely easy. Make changes directly to the objects fields, then call the object's ``save()`` method:: >>> p = Polls.objects.get(id__exact=15) >>> p.slug = "new_slug" >>> p.pub_date = datetime.datetime.now() >>> p.save() Creating new objects ==================== Creating new objects (i.e. ``INSERT``) is done by creating new instances of objects then calling save() on them:: >>> p = Poll(slug="eggs", ... question="How do you like your eggs?", ... pub_date=datetime.datetime.now(), ... expire_date=some_future_date) >>> p.save() Calling ``save()`` on an object with a primary key whose value is ``None`` signifies to Django that the object is new and should be inserted. Related objects are created using the ``create()`` convenience function on the descriptor Manager for relation:: >>> p.choice_set.create(choice="Over easy", votes=0) >>> p.choice_set.create(choice="Scrambled", votes=0) >>> p.choice_set.create(choice="Fertilized", votes=0) >>> p.choice_set.create(choice="Poached", votes=0) >>> p.choice_set.count() 4 Each of those ``create()`` methods is equivalent to (but much simpler than):: >>> c = Choice(poll_id=p.id, choice="Over easy", votes=0) >>> c.save() Note that when using the `create()`` method, you do not give any value for the ``id`` field, nor do you give a value for the field that stores the relation (``poll_id`` in this case). The ``create()`` method always returns the newly created object. Deleting objects ================ The delete method, conveniently, is named ``delete()``. This method immediately deletes the object and has no return value. Example:: >>> c.delete() Objects can also be deleted in bulk. Every Query Set has a ``delete()`` method that will delete all members of the query set. For example:: >>> Polls.objects.filter(pub_date__year=2005).delete() would bulk delete all Polls with a year of 2005. Note that ``delete()`` is the only Query Set method that is not exposed on the Manager itself. This is a safety mechanism to prevent you from accidentally requesting ``Polls.objects.delete()``, and deleting *all* the polls. If you *actually* want to delete all the objects, then you have to explicitly request a complete query set:: Polls.objects.all().delete() Comparing objects ================= To compare two model objects, just use the standard Python comparison operator, the double equals sign: ``==``. Behind the scenes, that compares the primary key values of two models. Using the ``Poll`` example above, the following two statements are equivalent:: some_poll == other_poll some_poll.id == other_poll.id If a model's primary key isn't called ID, no problem. Comparisons will always use the primary key, whatever it's called. For example, if a model's primary key field is called ``name``, these two statements are equivalent:: some_obj == other_obj some_obj.name == other_obj.name Extra instance methods ====================== In addition to ``save()``, ``delete()`` and all of the ``add_*`` and ``get_*`` related-object methods, a model object might get any or all of the following methods: get_FOO_display() ----------------- For every field that has ``choices`` set, the object will have a ``get_FOO_display()`` method, where ``FOO`` is the name of the field. This method returns the "human-readable" value of the field. For example, in the following model:: GENDER_CHOICES = ( ('M', 'Male'), ('F', 'Female'), ) class Person name = meta.CharField(maxlength=20) gender = meta.CharField(maxlength=1, choices=GENDER_CHOICES) ...each ``Person`` instance will have a ``get_gender_display()`` method. Example:: >>> p = Person(name='John', gender='M') >>> p.save() >>> p.gender 'M' >>> p.get_gender_display() 'Male' get_next_by_FOO(\**kwargs) and get_previous_by_FOO(\**kwargs) ------------------------------------------------------------- For every ``DateField`` and ``DateTimeField`` that does not have ``null=True``, the object will have ``get_next_by_FOO()`` and ``get_previous_by_FOO()`` methods, where ``FOO`` is the name of the field. This returns the next and previous object with respect to the date field, raising the appropriate ``*DoesNotExist`` exception when appropriate. Both methods accept optional keyword arguments, which should be in the format described in "Field lookups" above. Note that in the case of identical date values, these methods will use the ID as a fallback check. This guarantees that no records are skipped or duplicated. For a full example, see the `lookup API sample model_`. .. _lookup API sample model: http://www.djangoproject.com/documentation/models/lookup/ get_FOO_filename() ------------------ For every ``FileField``, the object will have a ``get_FOO_filename()`` method, where ``FOO`` is the name of the field. This returns the full filesystem path to the file, according to your ``MEDIA_ROOT`` setting. Note that ``ImageField`` is technically a subclass of ``FileField``, so every model with an ``ImageField`` will also get this method. get_FOO_url() ------------- For every ``FileField``, the object will have a ``get_FOO_url()`` method, where ``FOO`` is the name of the field. This returns the full URL to the file, according to your ``MEDIA_URL`` setting. If the value is blank, this method returns an empty string. get_FOO_size() -------------- For every ``FileField``, the object will have a ``get_FOO_filename()`` method, where ``FOO`` is the name of the field. This returns the size of the file, in bytes. (Behind the scenes, it uses ``os.path.getsize``.) save_FOO_file(filename, raw_contents) ------------------------------------- For every ``FileField``, the object will have a ``get_FOO_filename()`` method, where ``FOO`` is the name of the field. This saves the given file to the filesystem, using the given filename. If a file with the given filename already exists, Django adds an underscore to the end of the filename (but before the extension) until the filename is available. get_FOO_height() and get_FOO_width() ------------------------------------ For every ``ImageField``, the object will have ``get_FOO_height()`` and ``get_FOO_width()`` methods, where ``FOO`` is the name of the field. This returns the height (or width) of the image, as an integer, in pixels.