From 6103059592c6c1bd171977d26b76ee475175043c Mon Sep 17 00:00:00 2001 From: Filip Lajszczak Date: Sun, 25 Sep 2022 14:08:24 +0100 Subject: [PATCH] Fixed #34054 -- Created a new fixtures topic. Moved material from django-admin document into a new document, and added new material. Co-authored-by: Mariusz Felisiak --- docs/howto/initial-data.txt | 5 +- docs/ref/django-admin.txt | 153 ++++----------------------------- docs/ref/settings.txt | 4 +- docs/ref/signals.txt | 12 +-- docs/topics/db/fixtures.txt | 157 ++++++++++++++++++++++++++++++++++ docs/topics/db/index.txt | 1 + docs/topics/testing/tools.txt | 6 +- 7 files changed, 190 insertions(+), 148 deletions(-) create mode 100644 docs/topics/db/fixtures.txt diff --git a/docs/howto/initial-data.txt b/docs/howto/initial-data.txt index 08c55e1114..af2852cc7b 100644 --- a/docs/howto/initial-data.txt +++ b/docs/howto/initial-data.txt @@ -19,8 +19,9 @@ limitations `. Provide data with fixtures ========================== -You can also provide data using fixtures, however, this data isn't loaded -automatically, except if you use :attr:`.TransactionTestCase.fixtures`. +You can also provide data using :ref:`fixtures `, +however, this data isn't loaded automatically, except if you use +:attr:`.TransactionTestCase.fixtures`. A fixture is a collection of data that Django knows how to import into a database. The most straightforward way of creating a fixture if you've already diff --git a/docs/ref/django-admin.txt b/docs/ref/django-admin.txt index b2fd82262d..5e52fc282e 100644 --- a/docs/ref/django-admin.txt +++ b/docs/ref/django-admin.txt @@ -296,6 +296,11 @@ If no application name is provided, all installed applications will be dumped. The output of ``dumpdata`` can be used as input for :djadmin:`loaddata`. +When result of ``dumpdata`` is saved as a file, it can serve as a +:ref:`fixture ` for +:ref:`tests ` or as an +:ref:`initial data `. + Note that ``dumpdata`` uses the default manager on the model for selecting the records to dump. If you're using a :ref:`custom manager ` as the default manager and it filters some of the available records, not all of the @@ -480,7 +485,8 @@ If this option is provided, models are also created for database views. .. django-admin:: loaddata fixture [fixture ...] -Searches for and loads the contents of the named fixture into the database. +Searches for and loads the contents of the named +:ref:`fixture ` into the database. .. django-admin-option:: --database DATABASE @@ -508,135 +514,6 @@ Excludes loading the fixtures from the given applications and/or models (in the form of ``app_label`` or ``app_label.ModelName``). Use the option multiple times to exclude more than one app or model. -What's a "fixture"? -~~~~~~~~~~~~~~~~~~~ - -A *fixture* is a collection of files that contain the serialized contents of -the database. Each fixture has a unique name, and the files that comprise the -fixture can be distributed over multiple directories, in multiple applications. - -Django will search in three locations for fixtures: - -1. In the ``fixtures`` directory of every installed application -2. In any directory named in the :setting:`FIXTURE_DIRS` setting -3. In the literal path named by the fixture - -Django will load any and all fixtures it finds in these locations that match -the provided fixture names. - -If the named fixture has a file extension, only fixtures of that type -will be loaded. For example:: - - django-admin loaddata mydata.json - -would only load JSON fixtures called ``mydata``. The fixture extension -must correspond to the registered name of a -:ref:`serializer ` (e.g., ``json`` or ``xml``). - -If you omit the extensions, Django will search all available fixture types -for a matching fixture. For example:: - - django-admin loaddata mydata - -would look for any fixture of any fixture type called ``mydata``. If a fixture -directory contained ``mydata.json``, that fixture would be loaded -as a JSON fixture. - -The fixtures that are named can include directory components. These -directories will be included in the search path. For example:: - - django-admin loaddata foo/bar/mydata.json - -would search ``/fixtures/foo/bar/mydata.json`` for each installed -application, ``/foo/bar/mydata.json`` for each directory in -:setting:`FIXTURE_DIRS`, and the literal path ``foo/bar/mydata.json``. - -When fixture files are processed, the data is saved to the database as is. -Model defined :meth:`~django.db.models.Model.save` methods are not called, and -any :data:`~django.db.models.signals.pre_save` or -:data:`~django.db.models.signals.post_save` signals will be called with -``raw=True`` since the instance only contains attributes that are local to the -model. You may, for example, want to disable handlers that access -related fields that aren't present during fixture loading and would otherwise -raise an exception:: - - from django.db.models.signals import post_save - from .models import MyModel - - def my_handler(**kwargs): - # disable the handler during fixture loading - if kwargs['raw']: - return - ... - - post_save.connect(my_handler, sender=MyModel) - -You could also write a decorator to encapsulate this logic:: - - from functools import wraps - - def disable_for_loaddata(signal_handler): - """ - Decorator that turns off signal handlers when loading fixture data. - """ - @wraps(signal_handler) - def wrapper(*args, **kwargs): - if kwargs['raw']: - return - signal_handler(*args, **kwargs) - return wrapper - - @disable_for_loaddata - def my_handler(**kwargs): - ... - -Just be aware that this logic will disable the signals whenever fixtures are -deserialized, not just during ``loaddata``. - -Note that the order in which fixture files are processed is undefined. However, -all fixture data is installed as a single transaction, so data in -one fixture can reference data in another fixture. If the database backend -supports row-level constraints, these constraints will be checked at the -end of the transaction. - -The :djadmin:`dumpdata` command can be used to generate input for ``loaddata``. - -Compressed fixtures -~~~~~~~~~~~~~~~~~~~ - -Fixtures may be compressed in ``zip``, ``gz``, ``bz2``, ``lzma``, or ``xz`` -format. For example:: - - django-admin loaddata mydata.json - -would look for any of ``mydata.json``, ``mydata.json.zip``, ``mydata.json.gz``, -``mydata.json.bz2``, ``mydata.json.lzma``, or ``mydata.json.xz``. The first -file contained within a compressed archive is used. - -Note that if two fixtures with the same name but different -fixture type are discovered (for example, if ``mydata.json`` and -``mydata.xml.gz`` were found in the same fixture directory), fixture -installation will be aborted, and any data installed in the call to -``loaddata`` will be removed from the database. - -.. admonition:: MySQL with MyISAM and fixtures - - The MyISAM storage engine of MySQL doesn't support transactions or - constraints, so if you use MyISAM, you won't get validation of fixture - data, or a rollback if multiple transaction files are found. - -Database-specific fixtures -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you're in a multi-database setup, you might have fixture data that -you want to load onto one database, but not onto another. In this -situation, you can add a database identifier into the names of your fixtures. - -For example, if your :setting:`DATABASES` setting has a 'users' database -defined, name the fixture ``mydata.users.json`` or -``mydata.users.json.gz`` and the fixture will only be loaded when you -specify you want to load data into the ``users`` database. - .. _loading-fixtures-stdin: Loading fixtures from ``stdin`` @@ -656,6 +533,12 @@ For example:: django-admin dumpdata --format=json --database=test app_label.ModelName | django-admin loaddata --format=json --database=prod - +The :djadmin:`dumpdata` command can be used to generate input for ``loaddata``. + +.. seealso:: + + For more detail about fixtures see the :ref:`fixtures-explanation` topic. + ``makemessages`` ---------------- @@ -1629,11 +1512,11 @@ This is useful in a number of ways: * Let's say you're developing your Django application and have a "pristine" copy of a database that you'd like to interact with. You can dump your - database to a fixture (using the :djadmin:`dumpdata` command, explained - above), then use ``testserver`` to run your web application with that data. - With this arrangement, you have the flexibility of messing up your data - in any way, knowing that whatever data changes you're making are only - being made to a test database. + database to a :ref:`fixture ` (using the + :djadmin:`dumpdata` command, explained above), then use ``testserver`` to run + your web application with that data. With this arrangement, you have the + flexibility of messing up your data in any way, knowing that whatever data + changes you're making are only being made to a test database. Note that this server does *not* automatically detect changes to your Python source code (as :djadmin:`runserver` does). It does, however, detect changes to diff --git a/docs/ref/settings.txt b/docs/ref/settings.txt index 79520c3e2d..e1a2ead760 100644 --- a/docs/ref/settings.txt +++ b/docs/ref/settings.txt @@ -1665,8 +1665,8 @@ Monday and so on. Default: ``[]`` (Empty list) -List of directories searched for fixture files, in addition to the -``fixtures`` directory of each application, in search order. +List of directories searched for :ref:`fixture ` files, +in addition to the ``fixtures`` directory of each application, in search order. Note that these paths should use Unix-style forward slashes, even on Windows. diff --git a/docs/ref/signals.txt b/docs/ref/signals.txt index 2fdd215a8f..d9c07b415c 100644 --- a/docs/ref/signals.txt +++ b/docs/ref/signals.txt @@ -131,9 +131,9 @@ Arguments sent with this signal: ``raw`` A boolean; ``True`` if the model is saved exactly as presented - (i.e. when loading a fixture). One should not query/modify other - records in the database as the database might not be in a - consistent state yet. + (i.e. when loading a :ref:`fixture `). One should not + query/modify other records in the database as the database might not be in + a consistent state yet. ``using`` The database alias being used. @@ -164,9 +164,9 @@ Arguments sent with this signal: ``raw`` A boolean; ``True`` if the model is saved exactly as presented - (i.e. when loading a fixture). One should not query/modify other - records in the database as the database might not be in a - consistent state yet. + (i.e. when loading a :ref:`fixture `). One should not + query/modify other records in the database as the database might not be in + a consistent state yet. ``using`` The database alias being used. diff --git a/docs/topics/db/fixtures.txt b/docs/topics/db/fixtures.txt new file mode 100644 index 0000000000..b1425adc13 --- /dev/null +++ b/docs/topics/db/fixtures.txt @@ -0,0 +1,157 @@ +.. _fixtures-explanation: + +======== +Fixtures +======== + +.. seealso:: + + * :doc:`/howto/initial-data` + +What is a fixture? +================== + +A *fixture* is a collection of files that contain the serialized contents of +the database. Each fixture has a unique name, and the files that comprise the +fixture can be distributed over multiple directories, in multiple applications. + +How to produce a fixture? +========================= + +Fixtures can be generated by :djadmin:`manage.py dumpdata `. It's +also possible to generate custom fixtures by directly using +:doc:`serialization documentation ` tools or even by +handwriting them. + +What to use a fixture for? +========================== + +Fixtures can be used to pre-populate database with data for +:ref:`tests ` or to provide some :ref:`initial data +`. + +Were Django looks for fixtures? +=============================== + +Django will search in three locations for fixtures: + +1. In the ``fixtures`` directory of every installed application +2. In any directory named in the :setting:`FIXTURE_DIRS` setting +3. In the literal path named by the fixture + +Django will load any and all fixtures it finds in these locations that match +the provided fixture names. + +If the named fixture has a file extension, only fixtures of that type +will be loaded. For example:: + + django-admin loaddata mydata.json + +would only load JSON fixtures called ``mydata``. The fixture extension +must correspond to the registered name of a +:ref:`serializer ` (e.g., ``json`` or ``xml``). + +If you omit the extensions, Django will search all available fixture types +for a matching fixture. For example:: + + django-admin loaddata mydata + +would look for any fixture of any fixture type called ``mydata``. If a fixture +directory contained ``mydata.json``, that fixture would be loaded +as a JSON fixture. + +The fixtures that are named can include directory components. These +directories will be included in the search path. For example:: + + django-admin loaddata foo/bar/mydata.json + +would search ``/fixtures/foo/bar/mydata.json`` for each installed +application, ``/foo/bar/mydata.json`` for each directory in +:setting:`FIXTURE_DIRS`, and the literal path ``foo/bar/mydata.json``. + +How fixtures are saved to the database? +======================================= + +When fixture files are processed, the data is saved to the database as is. +Model defined :meth:`~django.db.models.Model.save` methods are not called, and +any :data:`~django.db.models.signals.pre_save` or +:data:`~django.db.models.signals.post_save` signals will be called with +``raw=True`` since the instance only contains attributes that are local to the +model. You may, for example, want to disable handlers that access +related fields that aren't present during fixture loading and would otherwise +raise an exception:: + + from django.db.models.signals import post_save + from .models import MyModel + + def my_handler(**kwargs): + # disable the handler during fixture loading + if kwargs['raw']: + return + ... + + post_save.connect(my_handler, sender=MyModel) + +You could also write a decorator to encapsulate this logic:: + + from functools import wraps + + def disable_for_loaddata(signal_handler): + """ + Decorator that turns off signal handlers when loading fixture data. + """ + @wraps(signal_handler) + def wrapper(*args, **kwargs): + if kwargs['raw']: + return + signal_handler(*args, **kwargs) + return wrapper + + @disable_for_loaddata + def my_handler(**kwargs): + ... + +Just be aware that this logic will disable the signals whenever fixtures are +deserialized, not just during ``loaddata``. + +Note that the order in which fixture files are processed is undefined. However, +all fixture data is installed as a single transaction, so data in +one fixture can reference data in another fixture. If the database backend +supports row-level constraints, these constraints will be checked at the +end of the transaction. + +Compressed fixtures +=================== + +Fixtures may be compressed in ``zip``, ``gz``, ``bz2``, ``lzma``, or ``xz`` +format. For example:: + + django-admin loaddata mydata.json + +would look for any of ``mydata.json``, ``mydata.json.zip``, ``mydata.json.gz``, +``mydata.json.bz2``, ``mydata.json.lzma``, or ``mydata.json.xz``. The first +file contained within a compressed archive is used. + +Note that if two fixtures with the same name but different +fixture type are discovered (for example, if ``mydata.json`` and +``mydata.xml.gz`` were found in the same fixture directory), fixture +installation will be aborted, and any data installed in the call to +``loaddata`` will be removed from the database. + +.. admonition:: MySQL with MyISAM and fixtures + + The MyISAM storage engine of MySQL doesn't support transactions or + constraints, so if you use MyISAM, you won't get validation of fixture + data, or a rollback if multiple transaction files are found. + +Database-specific fixtures +========================== + +If you're in a multi-database setup, you might have fixture data that +you want to load onto one database, but not onto another. In this +situation, you can add a database identifier into the names of your fixtures. + +For example, if your :setting:`DATABASES` setting has a ``users`` database +defined, name the fixture ``mydata.users.json`` or +``mydata.users.json.gz`` and the fixture will only be loaded when you +specify you want to load data into the ``users`` database. diff --git a/docs/topics/db/index.txt b/docs/topics/db/index.txt index 987bfefc94..67a71fd820 100644 --- a/docs/topics/db/index.txt +++ b/docs/topics/db/index.txt @@ -22,4 +22,5 @@ Generally, each model maps to a single database table. tablespaces optimization instrumentation + fixtures examples/index diff --git a/docs/topics/testing/tools.txt b/docs/topics/testing/tools.txt index 0eb6fee113..27692d313d 100644 --- a/docs/topics/testing/tools.txt +++ b/docs/topics/testing/tools.txt @@ -1101,7 +1101,7 @@ Fixture loading A test case for a database-backed website isn't much use if there isn't any data in the database. Tests are more readable and it's more maintainable to create objects using the ORM, for example in :meth:`TestCase.setUpTestData`, -however, you can also use fixtures. +however, you can also use :ref:`fixtures `. A fixture is a collection of data that Django knows how to import into a database. For example, if your site has user accounts, you might set up a @@ -1139,8 +1139,8 @@ Here's specifically what will happen: * Then, all the named fixtures are installed. In this example, Django will install any JSON fixture named ``mammals``, followed by any fixture named - ``birds``. See the :djadmin:`loaddata` documentation for more - details on defining and installing fixtures. + ``birds``. See the :ref:`fixtures-explanation` topic for more details on + defining and installing fixtures. For performance reasons, :class:`TestCase` loads fixtures once for the entire test class, before :meth:`~TestCase.setUpTestData`, instead of before each