feat(backend): add performance tests (#11017)

* feat(backend): add performance test

ref #11002

* feat(backend): add performance test (#486)

* chore(deps): bump the dependencies group across 1 directory with 2 updates (#11003)

* chore(deps): bump the dependencies group across 1 directory with 2 updates

Bumps the dependencies group with 2 updates in the /src/backend directory: [django-q2](https://github.com/GDay/django-q2) and [sentry-sdk](https://github.com/getsentry/sentry-python).


Updates `django-q2` from 1.8.0 to 1.9.0
- [Release notes](https://github.com/GDay/django-q2/releases)
- [Changelog](https://github.com/django-q2/django-q2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GDay/django-q2/compare/v1.8.0...v1.9.0)

Updates `sentry-sdk` from 2.46.0 to 2.47.0
- [Release notes](https://github.com/getsentry/sentry-python/releases)
- [Changelog](https://github.com/getsentry/sentry-python/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/sentry-python/compare/2.46.0...2.47.0)

---
updated-dependencies:
- dependency-name: django-q2
  dependency-version: 1.9.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
- dependency-name: sentry-sdk
  dependency-version: 2.47.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix style

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matthias Mair <code@mjmair.com>

* Rearrange python package installs in are metal setup  (#11005)

* Reorder pip installation steps in bare metal setup

* Reorder pip installation steps in bare metal setup

* remove unused lines

* Fix docs formatting (#11008)

* Remove prefetch_related from parametric data filter (#11007)

- Not required as we do not process the parameter fields in python

* [refactor] Generic status API (#11009)

* Fix docs formatting

* [refactor] cache custom states

- Generic state API endpoint executed  query for each state type
- We can run a single database query and cache these in memory
- Reduces query time by ~50%

* [refactor] Build list (#11010)

- Prefetch project_code
- Annotate parameter data

* Improve the documentation installation instructions. (#11011)

Co-authored-by: Mitch Davis <mjd@afork.com>

* [refactor] Improve primary_address annotation for Company API (#11006)

* Refactor primary_address annotation

- Remove SerializerMethodField
- Better cache introspection

* Allow address detail to be optional

* Refactor address caching

* Fix primary_address annotation

* Remove "address_count" field

- Pointless annotation which is not used anywhere

* Update API version

* Tweak docs page

* Tweak unit tests

* feat(backend): add performance test

ref #11002

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Michael <michael@buchmann.ruhr>
Co-authored-by: Oliver <oliver.henry.walters@gmail.com>
Co-authored-by: Mitch Davis <mjd+github@afork.com>
Co-authored-by: Mitch Davis <mjd@afork.com>

* add oidc perm

* fix run setup

* add gitignore

* pin action

* enable DB for test

* patch test detection

* move test argument into tasks

* seperate performance testing into own step

* add automigration

* update test

* Increase MAX_QUERY_TIME to 60 seconds

* use newer python for better prerformance / measurement options

* skip plugin install step

* add debug step

* add debug stmt

* make version import safe

* fix command

* more debugging

* move import

* rollback changes

* do full install

* rollback skip_plugins too

* hide version

* new debug try

* add more debug

* try 3.13

* try reinstalling the cffi

* reinstall cffi?

* reset debug

* rollback debug steos

* add initial tests

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Michael <michael@buchmann.ruhr>
Co-authored-by: Oliver <oliver.henry.walters@gmail.com>
Co-authored-by: Mitch Davis <mjd+github@afork.com>
Co-authored-by: Mitch Davis <mjd@afork.com>

.github/workflows/qc_checks.yaml

@@ -377,6 +377,39 @@ jobs:
           slug: inventree/InvenTree
           flags: backend
 
+  performance:
+    name: Tests - Performance
+    runs-on: ubuntu-24.04
+
+    needs: ["pre-commit", "paths-filter"]
+    if: needs.paths-filter.outputs.server == 'true' || needs.paths-filter.outputs.force == 'true'
+    permissions:
+      contents: read
+      id-token: write
+
+    env:
+      INVENTREE_DB_NAME: inventree_unit_test_db.sqlite
+      INVENTREE_DB_ENGINE: sqlite3
+      INVENTREE_PLUGINS_ENABLED: true
+      INVENTREE_CONSOLE_LOG: false
+      INVENTREE_AUTO_UPDATE: true
+
+    steps:
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # pin@v6.0.1
+        with:
+          persist-credentials: false
+      - name: Environment Setup
+        uses: ./.github/actions/setup
+        with:
+          apt-dependency: gettext poppler-utils
+          dev-install: true
+          update: true
+      - name: Performance Reporting
+        uses: CodSpeedHQ/action@346a2d8a8d9d38909abd0bc3d23f773110f076ad # pin@v4
+        with:
+          mode: simulation
+          run: inv dev.test --pytest
+
   postgres:
     name: Tests - DB [PostgreSQL]
     runs-on: ubuntu-24.04

.gitignore

@@ -116,3 +116,6 @@ api.yaml
 # web frontend (static files)
 src/backend/InvenTree/web/static
 InvenTree/web/static
+
+# performance test results
+.codspeed/

pyproject.toml

@@ -133,3 +133,9 @@ sections=["FUTURE","STDLIB","DJANGO","THIRDPARTY","FIRSTPARTY","LOCALFOLDER"]
 
 [tool.codespell]
 ignore-words-list = ["assertIn","SME","intoto","fitH"]
+
+[tool.pytest]
+django_find_project = false
+pythonpath = ["src/backend/InvenTree"]
+DJANGO_SETTINGS_MODULE = "InvenTree.settings"
+python_files = ["test*.py",]

src/backend/InvenTree/InvenTree/ready.py

@@ -30,7 +30,7 @@ def isAppLoaded(app_name: str) -> bool:
 
 def isInTestMode():
     """Returns True if the database is in testing mode."""
-    return 'test' in sys.argv
+    return 'test' in sys.argv or sys.argv[0].endswith('pytest')
 
 
 def isImportingData():

src/backend/InvenTree/InvenTree/settings.py

@@ -53,7 +53,7 @@ INVENTREE_BASE_URL = 'https://inventree.org'
 INVENTREE_NEWS_URL = f'{INVENTREE_BASE_URL}/news/feed.atom'
 
 # Determine if we are running in "test" mode e.g. "manage.py test"
-TESTING = 'test' in sys.argv or 'TESTING' in os.environ
+TESTING = 'test' in sys.argv or 'TESTING' in os.environ or 'pytest' in sys.argv
 
 if TESTING:
     # Use a weaker password hasher for testing (improves testing speed)

src/backend/InvenTree/InvenTree/unit_test.py

@@ -16,7 +16,7 @@ from django.contrib.auth import get_user_model
 from django.contrib.auth.models import Group, Permission, User
 from django.db import connections, models
 from django.http.response import StreamingHttpResponse
-from django.test import TestCase
+from django.test import TestCase, tag
 from django.test.utils import CaptureQueriesContext, override_settings
 from django.urls import reverse
 
@@ -824,3 +824,11 @@ class AdminTestCase(InvenTreeAPITestCase):
 def in_env_context(envs):
     """Patch the env to include the given dict."""
     return mock.patch.dict(os.environ, envs)
+
+
+@tag('performance_test')
+class InvenTreeAPIPerformanceTestCase(InvenTreeAPITestCase):
+    """Base class for InvenTree API performance tests."""
+
+    MAX_QUERY_COUNT = 50
+    MAX_QUERY_TIME = 60

src/backend/InvenTree/InvenTree/version.py

@@ -12,9 +12,6 @@ import sys
 from datetime import datetime as dt
 from datetime import timedelta as td
 
-import django
-from django.conf import settings
-
 from .api_version import INVENTREE_API_TEXT, INVENTREE_API_VERSION
 
 # InvenTree software version
@@ -230,6 +227,8 @@ def inventreeApiText(versions: int = 10, start_version: int = 0):
 
 def inventreeDjangoVersion():
     """Returns the version of Django library."""
+    import django
+
     return django.get_version()
 
 
@@ -296,6 +295,8 @@ def inventreePlatform():
 
 def inventreeDatabase():
     """Return the InvenTree database backend e.g. 'postgresql'."""
+    from django.conf import settings
+
     return settings.DB_ENGINE
 
 

src/backend/InvenTree/part/test_api.py

@@ -11,6 +11,7 @@ from django.db import connection
 from django.test.utils import CaptureQueriesContext
 from django.urls import reverse
 
+import pytest
 from PIL import Image
 from rest_framework.test import APIClient
 
@@ -21,7 +22,7 @@ from build.status_codes import BuildStatus
 from common.models import InvenTreeSetting, ParameterTemplate
 from company.models import Company, SupplierPart
 from InvenTree.config import get_testfolder_dir
-from InvenTree.unit_test import InvenTreeAPITestCase
+from InvenTree.unit_test import InvenTreeAPIPerformanceTestCase, InvenTreeAPITestCase
 from order.status_codes import PurchaseOrderStatusGroups
 from part.models import (
     BomItem,
@@ -3349,3 +3350,15 @@ class ParameterTests(PartAPITestBase):
 
         self.assertIn('export_format', fields)
         self.assertIn('export_plugin', fields)
+
+
+class PartApiPerformanceTest(PartAPITestBase, InvenTreeAPIPerformanceTestCase):
+    """Performance tests for the Part API."""
+
+    @pytest.mark.django_db
+    @pytest.mark.benchmark
+    def test_api_part_list(self):
+        """Test that Part API queries are performant."""
+        url = reverse('api-part-list')
+        response = self.get(url, expected_code=200)
+        self.assertGreater(len(response.data), 13)

src/backend/InvenTree/stock/test_api.py

@@ -9,6 +9,7 @@ from django.contrib.contenttypes.models import ContentType
 from django.core.exceptions import ValidationError
 from django.urls import reverse
 
+import pytest
 from djmoney.money import Money
 from rest_framework import status
 
@@ -17,7 +18,7 @@ import company.models
 import part.models
 from common.models import InvenTreeCustomUserStateModel, InvenTreeSetting
 from common.settings import set_global_setting
-from InvenTree.unit_test import InvenTreeAPITestCase
+from InvenTree.unit_test import InvenTreeAPIPerformanceTestCase, InvenTreeAPITestCase
 from part.models import Part, PartTestTemplate
 from stock.models import (
     StockItem,
@@ -2549,3 +2550,15 @@ class StockMetadataAPITest(InvenTreeAPITestCase):
             'api-stock-item-metadata': StockItem,
         }.items():
             self.metatester(apikey, model)
+
+
+class StockApiPerformanceTest(StockAPITestCase, InvenTreeAPIPerformanceTestCase):
+    """Performance tests for the Stock API."""
+
+    @pytest.mark.django_db
+    @pytest.mark.benchmark
+    def test_api_stock_list(self):
+        """Test that Stock API queries are performant."""
+        url = reverse('api-stock-list')
+        response = self.get(url, expected_code=200)
+        self.assertGreater(len(response.data), 13)

src/backend/requirements-dev.in

@@ -14,3 +14,5 @@ ty                                      # type checking
 django-types                            # typing
 django-stubs                            # typing
 requests-mock                           # Mock requests for unit tests
+pytest-codspeed                         # Performance testing with Codspeed
+pytest-django                           # Pytest support for Django (for benchnmarking)

src/backend/requirements-dev.txt

@@ -104,6 +104,7 @@ cffi==2.0.0 \
     # via
     #   -c src/backend/requirements.txt
     #   cryptography
+    #   pytest-codspeed
 cfgv==3.5.0 \
     --hash=sha256:a8dc6b26ad22ff227d2634a65cb388215ce6cc96bbcc5cfde7641ae87e8dacc0 \
     --hash=sha256:d5b1034354820651caa73ede66a6294d6e95c1b00acc5e9b098e917404669132
@@ -439,10 +440,22 @@ idna==3.11 \
     # via
     #   -c src/backend/requirements.txt
     #   requests
+iniconfig==2.3.0 \
+    --hash=sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730 \
+    --hash=sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12
+    # via pytest
 isort==7.0.0 \
     --hash=sha256:1bcabac8bc3c36c7fb7b98a76c8abb18e0f841a3ba81decac7691008592499c1 \
     --hash=sha256:5513527951aadb3ac4292a41a16cbc50dd1642432f5e8c20057d414bdafb4187
     # via -r src/backend/requirements-dev.in
+markdown-it-py==4.0.0 \
+    --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \
+    --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3
+    # via rich
+mdurl==0.1.2 \
+    --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \
+    --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba
+    # via markdown-it-py
 nodeenv==1.9.1 \
     --hash=sha256:6ec12890a2dab7946721edbfbcd91f3319c6ccc9aec47be7c7e6b7011ee6645f \
     --hash=sha256:ba11c9782d29c27c70ffbdda2d7415098754709be8a7056d79a737cd901155c9
@@ -453,6 +466,7 @@ packaging==25.0 \
     # via
     #   -c src/backend/requirements.txt
     #   build
+    #   pytest
 pdfminer-six==20251107 \
     --hash=sha256:5fb0c553799c591777f22c0c72b77fc2522d7d10c70654e25f4c5f1fd996e008 \
     --hash=sha256:c09df33e4cbe6b26b2a79248a4ffcccafaa5c5d39c9fff0e6e81567f165b5401
@@ -471,6 +485,10 @@ platformdirs==4.5.0 \
     # via
     #   -c src/backend/requirements.txt
     #   virtualenv
+pluggy==1.6.0 \
+    --hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \
+    --hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746
+    # via pytest
 pre-commit==4.5.0 \
     --hash=sha256:25e2ce09595174d9c97860a95609f9f852c0614ba602de3561e267547f2335e1 \
     --hash=sha256:dc5a065e932b19fc1d4c653c6939068fe54325af8e741e74e88db4d28a4dd66b
@@ -481,12 +499,46 @@ pycparser==2.23 \
     # via
     #   -c src/backend/requirements.txt
     #   cffi
+pygments==2.19.2 \
+    --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \
+    --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b
+    # via
+    #   pytest
+    #   rich
 pyproject-hooks==1.2.0 \
     --hash=sha256:1e859bd5c40fae9448642dd871adf459e5e2084186e8d2c2a79a824c970da1f8 \
     --hash=sha256:9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913
     # via
     #   build
     #   pip-tools
+pytest==9.0.2 \
+    --hash=sha256:711ffd45bf766d5264d487b917733b453d917afd2b0ad65223959f59089f875b \
+    --hash=sha256:75186651a92bd89611d1d9fc20f0b4345fd827c41ccd5c299a868a05d70edf11
+    # via
+    #   pytest-codspeed
+    #   pytest-django
+pytest-codspeed==4.2.0 \
+    --hash=sha256:04b5d0bc5a1851ba1504d46bf9d7dbb355222a69f2cd440d54295db721b331f7 \
+    --hash=sha256:0881a736285f33b9a8894da8fe8e1775aa1a4310226abe5d1f0329228efb680c \
+    --hash=sha256:238e17abe8f08d8747fa6c7acff34fefd3c40f17a56a7847ca13dc8d6e8c6009 \
+    --hash=sha256:23a0c0fbf8bb4de93a3454fd9e5efcdca164c778aaef0a9da4f233d85cb7f5b8 \
+    --hash=sha256:2de87bde9fbc6fd53f0fd21dcf2599c89e0b8948d49f9bad224edce51c47e26b \
+    --hash=sha256:309b4227f57fcbb9df21e889ea1ae191d0d1cd8b903b698fdb9ea0461dbf1dfe \
+    --hash=sha256:50794dabea6ec90d4288904452051e2febace93e7edf4ca9f2bce8019dd8cd37 \
+    --hash=sha256:609828b03972966b75b9b7416fa2570c4a0f6124f67e02d35cd3658e64312a7b \
+    --hash=sha256:684fcd9491d810ded653a8d38de4835daa2d001645f4a23942862950664273f8 \
+    --hash=sha256:72aab8278452a6d020798b9e4f82780966adb00f80d27a25d1274272c54630d5 \
+    --hash=sha256:748411c832147bfc85f805af78a1ab1684f52d08e14aabe22932bbe46c079a5f \
+    --hash=sha256:7d4fefbd4ae401e2c60f6be920a0be50eef0c3e4a1f0a1c83962efd45be38b39 \
+    --hash=sha256:95aeb2479ca383f6b18e2cc9ebcd3b03ab184980a59a232aea6f370bbf59a1e3 \
+    --hash=sha256:a0ebd87f2a99467a1cfd8e83492c4712976e43d353ee0b5f71cbb057f1393aca \
+    --hash=sha256:dbbb2d61b85bef8fc7e2193f723f9ac2db388a48259d981bbce96319043e9830 \
+    --hash=sha256:e81bbb45c130874ef99aca97929d72682733527a49f84239ba575b5cb843bab0
+    # via -r src/backend/requirements-dev.in
+pytest-django==4.11.1 \
+    --hash=sha256:1b63773f648aa3d8541000c26929c1ea63934be1cfa674c76436966d73fe6a10 \
+    --hash=sha256:a949141a1ee103cb0e7a20f1451d355f83f5e4a5d07bdd4dcfdd1fd0ff227991
+    # via -r src/backend/requirements-dev.in
 pyyaml==6.0.3 \
     --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \
     --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \
@@ -574,6 +626,10 @@ requests-mock==1.12.1 \
     --hash=sha256:b1e37054004cdd5e56c84454cc7df12b25f90f382159087f4b6915aaeef39563 \
     --hash=sha256:e9e12e333b525156e82a3c852f22016b9158220d2f47454de9cae8a77d371401
     # via -r src/backend/requirements-dev.in
+rich==14.2.0 \
+    --hash=sha256:73ff50c7c0c1c77c8243079283f4edb376f0f6442433aecb8ce7e6d0b92d1fe4 \
+    --hash=sha256:76bc51fe2e57d2b1be1f96c524b890b816e334ab4c1e45888799bfaab0021edd
+    # via pytest-codspeed
 setuptools==80.9.0 \
     --hash=sha256:062d34222ad13e0cc312a4c02d73f059e86a4acbfbdea8f8f76b28c99f306922 \
     --hash=sha256:f36b47402ecde768dbfafc46e8e4207b4360c654f1f3bb84475f0a28628fb19c

tasks.py

@@ -1284,6 +1284,7 @@ def test_translations(c):
         'coverage': 'Run code coverage analysis (requires coverage package)',
         'translations': 'Compile translations before running tests',
         'keepdb': 'Keep the test database after running tests (default = False)',
+        'pytest': 'Use pytest to run tests',
     }
 )
 def test(
@@ -1296,6 +1297,7 @@ def test(
     coverage=False,
     translations=False,
     keepdb=False,
+    pytest=False,
 ):
     """Run unit-tests for InvenTree codebase.
 
@@ -1341,10 +1343,16 @@ def test(
     else:
         cmd += ' --exclude-tag migration_test'
 
+    cmd += ' --exclude-tag performance_test'
+
     if coverage:
         # Run tests within coverage environment, and generate report
         run(c, f'coverage run {manage_py_path()} {cmd}')
         run(c, 'coverage xml -i')
+    elif pytest:
+        # Use pytest to run the tests
+        migrate(c)
+        run(c, f'pytest {manage_py_path().parent.parent} --codspeed')
     else:
         # Run simple test runner, without coverage
         manage(c, cmd, pty=pty)
@@ -1529,6 +1537,13 @@ def version(c):
         get_static_dir,
     )
 
+    def get_value(fnc):
+        """Helper function to safely get value from function, catching import exceptions."""
+        try:
+            return fnc()
+        except (ModuleNotFoundError, ImportError):
+            return wrap_color('ENVIRONMENT ERROR', '91')
+
     # Gather frontend version information
     _, node, yarn = node_available(versions=True)
 
@@ -1561,17 +1576,17 @@ Invoke Tool {invoke_path}
 
 Installation paths:
 Base        {local_dir()}
-Config      {get_config_file()}
+Config      {get_value(get_config_file)}
-Plugin File {get_plugin_file() or NOT_SPECIFIED}
+Plugin File {get_value(get_plugin_file) or NOT_SPECIFIED}
-Media       {get_media_dir(error=False) or NOT_SPECIFIED}
+Media       {get_value(lambda: get_media_dir(error=False)) or NOT_SPECIFIED}
-Static      {get_static_dir(error=False) or NOT_SPECIFIED}
+Static      {get_value(lambda: get_static_dir(error=False)) or NOT_SPECIFIED}
-Backup      {get_backup_dir(error=False) or NOT_SPECIFIED}
+Backup      {get_value(lambda: get_backup_dir(error=False)) or NOT_SPECIFIED}
 
 Versions:
 InvenTree   {InvenTreeVersion.inventreeVersion()}
 API         {InvenTreeVersion.inventreeApiVersion()}
 Python      {python_version()}
-Django      {InvenTreeVersion.inventreeDjangoVersion()}
+Django      {get_value(InvenTreeVersion.inventreeDjangoVersion)}
 Node        {node if node else NA}
 Yarn        {yarn if yarn else NA}