1
0
mirror of https://github.com/django/django.git synced 2025-10-23 21:59:11 +00:00

Fixed #1142 -- Added multiple database support.

This monster of a patch is the result of Alex Gaynor's 2009 Google Summer of Code project.
Congratulations to Alex for a job well done.

Big thanks also go to:
 * Justin Bronn for keeping GIS in line with the changes,
 * Karen Tracey and Jani Tiainen for their help testing Oracle support
 * Brett Hoerner, Jon Loyens, and Craig Kimmerer for their feedback.
 * Malcolm Treddinick for his guidance during the GSoC submission process.
 * Simon Willison for driving the original design process
 * Cal Henderson for complaining about ponies he wanted.

... and everyone else too numerous to mention that helped to bring this feature into fruition.

git-svn-id: http://code.djangoproject.com/svn/django/trunk@11952 bcc190cf-cafb-0310-a4f2-bffc1f526a37
This commit is contained in:
Russell Keith-Magee
2009-12-22 15:18:51 +00:00
parent 7ef212af14
commit ff60c5f9de
231 changed files with 7860 additions and 5668 deletions

View File

@@ -16,3 +16,4 @@ model maps to a single database table.
managers
sql
transactions
multi-db

265
docs/topics/db/multi-db.txt Normal file
View File

@@ -0,0 +1,265 @@
.. _topics-db-multi-db:
==================
Multiple Databases
==================
.. versionadded:: 1.2
This topic guide describes Django's support for interacting with multiple
databases. Most of the rest of Django's documentation assumes you are
interacting with a single database. If you want to interact with multiple
databases, some additional steps must be taken.
Defining your databases
=======================
The first step to using more than one database with Django is to tell
Django about the database servers you'll be using. This is done using
the :setting:`DATABASES` setting. This setting maps database aliases,
which are a way to refer to a specific database throughout Django, to
a dictionary of settings for that specific connection. The settings in
the inner dictionaries are described fully in the :setting:`DATABASES`
documentation.
Regardless of how many databases you have, you *must* have a database
named ``'default'``. Any additional databases you have can have
whatever alias you choose.
The following is an example ``settings.py`` snippet defining two
databases - a default Postgres database, and a MySQL database called
``users``::
DATABASES = {
'default': {
'NAME': 'app_data',
'BACKEND': 'django.db.backends.postgres_psycopg2',
'USER': 'postgres_user',
'PASSWORD': 's3krit'
},
'users': {
'NAME': 'user_data'
'BACKEND': 'django.db.backends.mysql',
'USER': 'mysql_user',
'PASSWORD': 'priv4te'
}
}
If you attempt to access a database that you haven't defined in your
:setting:`DATABASES` setting then Django will raise a
``django.db.utils.ConnectionDoesNotExist`` exception.
Selecting a database for a ``QuerySet``
=======================================
It is possible to select the database for a ``QuerySet`` at any point
during it's construction. To choose the database that a query will be
preformed against simply call the ``using()`` method on the
``QuerySet``. ``using()`` takes a single argument: the alias of the
database on which you want to run the query. For example::
# This will run on the 'default' database...
>>> Author.objects.all()
# So will this...
>>> Author.objects.using('default').all()
# This will run on the 'other' database
>>> Author.objects.using('other').all()
Select a database to save to
============================
To choose what database to save a model to, provide a ``using`` keyword
argument to ``Model.save()``. For example if you had a user model that you
wanted to save to the ``'legacy_users'`` database you would save the user
by calling::
>>> user_obj.save(using='legacy_users')
Moving an object from one database to another
---------------------------------------------
If you have saved an instance to one database, it might be tempting to use
``save(using=...)`` as a way to migrate the instance to a new database. However,
if you don't take appropriate steps, this could have some unexpected consequences.
Consider the following example::
>>> p = Person(name='Fred')
>>> p.save(using='first') # (1)
# some other processing ...
>>> p.save(using='second') # (2)
In statement 1, a new Person object is saved to the ``first``
database. At this time, ``p`` doesn't have a primary key, so Django
issues a SQL ``INSERT`` statement. This creates a primary key, and
Django assigns that primary key to ``p``.
When the save occurs in statement 2, ``p`` already has a primary key
value, and Django will attempt to use that primary key on the new
database. If the primary key value isn't in use in the ``second``
database, then you won't have any problems -- the object will be
copied to the new databse.
However, if the primary key of ``p`` is already in use on the
``second`` database, the existing object on the ``second`` database
will be lost when ``p`` is saved.
There are two ways to avoid this outcome. Firstly, you can clear the
primary key of the instance. If an object has no primary key, Django
will treat it as a new object, avoiding any loss of data on the
``second`` database::
>>> p = Person(name='Fred')
>>> p.save(using='first')
# some other processing ...
>>> p.pk = None # Clear the PK
>>> p.save(using='second') # Write a completely new object
Secondly, you can use the ``force_insert`` option to ``save()`` to ensure that
Django does a SQL ``INSERT``::
>>> p = Person(name='Fred')
>>> p.save(using='first')
# some other processing ...
>>> p.save(using='second', force_insert=True)
This will ensure that the person named ``Fred`` will have the same
primary key on both databases. If that primary key is already in use
when you try to save onto the ``second`` database, an error will be
raised.
Select a database to delete from
================================
By default, a call to delete an existing object will be executed on the
same database that was used to retrieve the object in the first place::
>>> user_obj = User.objects.using('legacy_users').get(username='fred')
>>> user_obj.delete() # will delete from the `legacy_users` database
If you want to specify the database from which a model will be
deleted, you can use a ``using`` keyword argument to the
``Model.delete()`` method. This argument is analogous to the ``using``
keyword argument to ``save()``. For example if you were migrating a
user from the ``'legacy_users'`` database to the ``'new_users'``
database you might use the commands::
>>> user_obj.save(using='new_users')
>>> user_obj.delete(using='legacy_users')
Using ``Managers`` with multiple databases
==========================================
When you call ``using()`` Django returns a ``QuerySet`` that will be
evaluated against that database. However, sometimes you want to direct
a manager to use a specific database chain ``using()``. If you call
``using()``, you won't have access to any of the methods on the
manager.
To overcome this limitation, managers provide a ``db_manager()``
method. This method returns a copy of the *manager* bound to that
specific database. So, if you want to load an object using it's
natural key (using the ``get_by_natural_key()`` method on the manager,
you can call::
>>> Book.objects.db_mamanger("other").get_by_natural_key(...)
If you are overriding ``get_query_set()`` on your manager you must be sure to
either, call the method on the parent (using ``super()``), or do the
appropriate handling of the ``_db`` attribute on the manager. For example if
you wanted to return a custom ``QuerySet`` class from the ``get_query_set``
method you could do this::
class MyManager(models.Manager):
...
def get_query_set(self):
qs = CustomQuerySet(self.model)
if self._db is not None:
qs = qs.using(self._db)
return qs
Exposing multiple databases in Django's admin interface
=======================================================
Django's admin doesn't have any explicit support for multiple
databases. If you want to provide an admin interface for a model on a
database other than ``default``, you need to write custom
:class:`~django.contrib.admin.ModelAdmin` classes that will direct the
admin to use a specific database for content.
There are four methods that require customization on a ModelAdmin
object::
class MultiDBModelAdmin(admin.ModelAdmin):
# A handy constant for the name of the alternate database
using = 'other'
def save_model(self, request, obj, form, change):
# Tell Django to save objects to the 'other' database
obj.save(using=self.using)
def queryset(self, request):
# Tell Django to look for objects on the 'other' database
return super(MultiDBModelAdmin, self).queryset(request).using(self.using)
def formfield_for_foreignkey(self, db_field, request=None, **kwargs):
# Tell Django to populate ForeignKey widgets using a query
# on the 'other' database
return super(MultiDBModelAdmin, self).formfield_for_foreignkey(db_field, request=request, using=self.using, **kwargs)
def formfield_for_manytomany(self, db_field, request=None, **kwargs):
# Tell Django to populate ManyToMany widgets using a query
# on the 'other' database
return super(MultiDBModelAdmin, self).formfield_for_manytomany(db_field, request=request, using=self.using, **kwargs)
The implementation provided here implements a multi-db strategy where
all objects of a given type are stored on a specific database (e.g.,
all ``User`` objects are on the ``other`` database). If your usage of
multi-db is more complex, your ModelAdmin will need to reflect that
strategy.
Inlines can be handled in a similar fashion -- they require just three
customized methods::
class MultiDBTabularInline(admin.TabularInline):
using = 'other'
def queryset(self, request):
# Tell Django to look for inline objects on the 'other' database
return super(MultiDBTabularInline, self).queryset(request).using(self.using)
def formfield_for_foreignkey(self, db_field, request=None, **kwargs):
# Tell Django to populate ForeignKey widgets using a query
# on the 'other' database
return super(MultiDBTabularInline, self).formfield_for_foreignkey(db_field, request=request, using=self.using, **kwargs)
def formfield_for_manytomany(self, db_field, request=None, **kwargs):
# Tell Django to populate ManyToMany widgets using a query
# on the 'other' database
return super(MultiDBTabularInline, self).formfield_for_manytomany(db_field, request=request, using=self.using, **kwargs)
Once you have written your model admin definitions, they can be
registered with any Admin instance::
from django.contrib import admin
# Specialize the multi-db admin objects for use with specific models
class BookInline(MultiDBTabularInline):
model = Book
class PublisherAdmin(MultiDBModelAdmin):
inlines = [BookInline]
admin.site.register
admin.site.register(Author, MultiDBModelAdmin)
admin.site.register(Publisher, PublisherAdmin)
othersite = admin.Site('othersite')
othersite.register(Publisher, MultiDBModelAdmin)
This example sets up two admin sites. On the first site, the
``Author`` and ``Publisher`` objects are exposed; ``Publisher``
objects have an tabular inline showing books published by that
publisher. The second site exposes just publishers, without the
inlines.

View File

@@ -23,7 +23,7 @@ Performing raw queries
The ``raw()`` manager method can be used to perform raw SQL queries that
return model instances:
.. method:: Manager.raw(query, params=None, translations=None)
.. method:: Manager.raw(raw_query, params=None, translations=None)
This method method takes a raw SQL query, executes it, and returns model
instances.

View File

@@ -56,7 +56,10 @@ Controlling transaction management in views
For most people, implicit request-based transactions work wonderfully. However,
if you need more fine-grained control over how transactions are managed, you
can use Python decorators to change the way transactions are handled by a
particular view function.
particular view function. All of the decorators take an option ``using``
parameter which should be the alias for a database connection for which the
behavior applies to. If no alias is specified then the ``"default"`` database
is used.
.. note::
@@ -79,9 +82,14 @@ Example::
def viewfunc(request):
....
@transaction.autocommit(using="my_other_database")
def viewfunc2(request):
....
Within ``viewfunc()``, transactions will be committed as soon as you call
``model.save()``, ``model.delete()``, or any other function that writes to the
database.
database. ``viewfunc2()`` will have this same behavior, but for the
``"my_other_database"`` connection.
``django.db.transaction.commit_on_success``
-------------------------------------------
@@ -95,6 +103,10 @@ all the work done in a function::
def viewfunc(request):
....
@transaction.commit_on_success(using="my_other_database")
def viewfunc2(request):
....
If the function returns successfully, then Django will commit all work done
within the function at that point. If the function raises an exception, though,
Django will roll back the transaction.
@@ -127,6 +139,10 @@ Manual transaction management looks like this::
else:
transaction.commit()
@transaction.commit_manually(using="my_other_database")
def viewfunc2(request):
....
.. admonition:: An important note to users of earlier Django releases:
The database ``connection.commit()`` and ``connection.rollback()`` methods
@@ -169,21 +185,25 @@ issue a rollback, the entire transaction is rolled back. Savepoints provide
the ability to perform a fine-grained rollback, rather than the full rollback
that would be performed by ``transaction.rollback()``.
Each of these functions takes a ``using`` argument which should be the name of
a database for which the behavior applies. If no ``using`` argument is
provided then the ``"default"`` database is used.
Savepoints are controlled by three methods on the transaction object:
.. method:: transaction.savepoint()
.. method:: transaction.savepoint(using=None)
Creates a new savepoint. This marks a point in the transaction that
is known to be in a "good" state.
Returns the savepoint ID (sid).
.. method:: transaction.savepoint_commit(sid)
.. method:: transaction.savepoint_commit(sid, using=None)
Updates the savepoint to include any operations that have been performed
since the savepoint was created, or since the last commit.
.. method:: transaction.savepoint_rollback(sid)
.. method:: transaction.savepoint_rollback(sid, using=None)
Rolls the transaction back to the last point at which the savepoint was
committed.

View File

@@ -24,14 +24,6 @@ To enable session functionality, do the following:
The default ``settings.py`` created by ``django-admin.py startproject`` has
``SessionMiddleware`` activated.
* Add ``'django.contrib.sessions'`` to your ``INSTALLED_APPS`` setting,
and run ``manage.py syncdb`` to install the single database table
that stores session data.
.. versionchanged:: 1.0
This step is optional if you're not using the database session backend;
see `configuring the session engine`_.
If you don't want to use sessions, you might as well remove the
``SessionMiddleware`` line from ``MIDDLEWARE_CLASSES`` and ``'django.contrib.sessions'``
from your ``INSTALLED_APPS``. It'll save you a small bit of overhead.
@@ -46,6 +38,22 @@ By default, Django stores sessions in your database (using the model
some setups it's faster to store session data elsewhere, so Django can be
configured to store session data on your filesystem or in your cache.
Using database-backed sessions
------------------------------
If you want to use a database-backed session, you need to add
``'django.contrib.sessions'`` to your ``INSTALLED_APPS`` setting.
If you want to store your session data on a database other than ``default``
alias, you should set the :setting:`SESSION_DB_ALIAS` setting.
Once you have configured your installation, run ``manage.py syncdb``
to install the single database table that stores session data.
.. versionadded:: 1.2
The :setting:`SESSION_DB_ALIAS` setting was added in Django 1.2. It
is not required in earlier versions.
Using cached sessions
---------------------
@@ -86,6 +94,9 @@ disregards persistence. In most cases, the ``cached_db`` backend will be fast
enough, but if you need that last bit of performance, and are willing to let
session data be expunged from time to time, the ``cache`` backend is for you.
If you use the ``cached_db`` session backend, you also need to follow the
configuration instructions for the `using database-backed sessions`_.
Using file-based sessions
-------------------------
@@ -97,6 +108,7 @@ to output from ``tempfile.gettempdir()``, most likely ``/tmp``) to control
where Django stores session files. Be sure to check that your Web server has
permissions to read and write to this location.
Using sessions in views
=======================

View File

@@ -278,33 +278,35 @@ The test database
-----------------
Tests that require a database (namely, model tests) will not use your "real"
(production) database. A separate, blank database is created for the tests.
(production) database. Separate, blank databases are created for the tests.
Regardless of whether the tests pass or fail, the test database is destroyed
Regardless of whether the tests pass or fail, the test databases are destroyed
when all the tests have been executed.
By default this test database gets its name by prepending ``test_`` to the
value of the :setting:`DATABASE_NAME` setting. When using the SQLite database
engine the tests will by default use an in-memory database (i.e., the database
will be created in memory, bypassing the filesystem entirely!). If you want to
use a different database name, specify the :setting:`TEST_DATABASE_NAME`
setting.
By default the test databases get their names by prepending ``test_``
to the value of the :setting:`NAME`` settings for the databased
defined in :setting:`DATABASES`. When using the SQLite database engine
the tests will by default use an in-memory database (i.e., the
database will be created in memory, bypassing the filesystem
entirely!). If you want to use a different database name, specify
``TEST_NAME`` in the dictionary for any given database in
:setting:`DATABASES`.
Aside from using a separate database, the test runner will otherwise use all of
the same database settings you have in your settings file:
:setting:`DATABASE_ENGINE`, :setting:`DATABASE_USER`, :setting:`DATABASE_HOST`,
etc. The test database is created by the user specified by
:setting:`DATABASE_USER`, so you'll need to make sure that the given user
account has sufficient privileges to create a new database on the system.
Aside from using a separate database, the test runner will otherwise
use all of the same database settings you have in your settings file:
:setting:`ENGINE`, :setting:`USER`, :setting:`HOST`, etc. The test
database is created by the user specified by ``USER``, so you'll need
to make sure that the given user account has sufficient privileges to
create a new database on the system.
.. versionadded:: 1.0
For fine-grained control over the
character encoding of your test database, use the
:setting:`TEST_DATABASE_CHARSET` setting. If you're using MySQL, you can also
use the :setting:`TEST_DATABASE_COLLATION` setting to control the particular
collation used by the test database. See the :ref:`settings documentation
<ref-settings>` for details of these advanced settings.
For fine-grained control over the character encoding of your test
database, use the :setting:`TEST_CHARSET` option. If you're using
MySQL, you can also use the :setting:`TEST_COLLATION` option to
control the particular collation used by the test database. See the
:ref:`settings documentation <ref-settings>` for details of these
advanced settings.
Other test conditions
---------------------
@@ -1037,6 +1039,39 @@ URLconf for the duration of the test case.
.. _emptying-test-outbox:
Multi-database support
~~~~~~~~~~~~~~~~~~~~~~
.. attribute:: TestCase.multi_db
.. versionadded:: 1.2
Django sets up a test database corresponding to every database that is
defined in the :setting:``DATABASES`` definition in your settings
file. However, a big part of the time taken to run a Django TestCase
is consumed by the call to ``flush`` that ensures that you have a
clean database at the start of each test run. If you have multiple
databases, multiple flushes are required (one for each database),
which can be a time consuming activity -- especially if your tests
don't need to test multi-database activity.
As an optimization, Django only flushes the ``default`` database at
the start of each test run. If your setup contains multiple databases,
and you have a test that requires every database to be clean, you can
use the ``multi_db`` attribute on the test suite to request a full
flush.
For example::
class TestMyViews(TestCase):
multi_db = True
def testIndexPageView(self):
call_some_test_code()
This test case will flush *all* the test databases before running
``testIndexPageView``.
Emptying the test outbox
~~~~~~~~~~~~~~~~~~~~~~~~
@@ -1251,16 +1286,17 @@ utility methods in the ``django.test.utils`` module.
.. function:: setup_test_environment()
Performs any global pre-test setup, such as the installing the
instrumentation of the template rendering system and setting up the dummy
``SMTPConnection``.
instrumentation of the template rendering system and setting up
the dummy ``SMTPConnection``.
.. function:: teardown_test_environment()
Performs any global post-test teardown, such as removing the black magic
hooks into the template system and restoring normal e-mail services.
Performs any global post-test teardown, such as removing the black
magic hooks into the template system and restoring normal e-mail
services.
The creation module of the database backend (``connection.creation``) also
provides some utilities that can be useful during testing.
The creation module of the database backend (``connection.creation``)
also provides some utilities that can be useful during testing.
.. function:: create_test_db(verbosity=1, autoclobber=False)
@@ -1268,27 +1304,29 @@ provides some utilities that can be useful during testing.
``verbosity`` has the same behavior as in ``run_tests()``.
``autoclobber`` describes the behavior that will occur if a database with
the same name as the test database is discovered:
``autoclobber`` describes the behavior that will occur if a
database with the same name as the test database is discovered:
* If ``autoclobber`` is ``False``, the user will be asked to approve
destroying the existing database. ``sys.exit`` is called if the user
does not approve.
* If ``autoclobber`` is ``False``, the user will be asked to
approve destroying the existing database. ``sys.exit`` is
called if the user does not approve.
* If autoclobber is ``True``, the database will be destroyed without
consulting the user.
* If autoclobber is ``True``, the database will be destroyed
without consulting the user.
Returns the name of the test database that it created.
``create_test_db()`` has the side effect of modifying
``settings.DATABASE_NAME`` to match the name of the test database.
``create_test_db()`` has the side effect of modifying the value of
:setting:`NAME` in :setting:`DATABASES` to match the name of the test
database.
.. versionchanged:: 1.0
``create_test_db()`` now returns the name of the test database.
.. function:: destroy_test_db(old_database_name, verbosity=1)
Destroys the database whose name is in the :setting:`DATABASE_NAME` setting
and restores the value of :setting:`DATABASE_NAME` to the provided name.
Destroys the database whose name is in stored in :setting:`NAME` in the
:setting:`DATABASES`, and sets :setting:`NAME` to use the
provided name.
``verbosity`` has the same behavior as in ``run_tests()``.