inventree/InvenTree
2
0
Fork 0
mirror of https://github.com/inventree/InvenTree.git synced 2026-02-25 16:17:58 +00:00
Code Activity

feat(backend): ensure restore of backups only works in correct enviroments (#11372)

Browse Source 
* [FR] ensure restore of backups only works in correct enviroments
Fixes #11214

* update PR nbr

* fix wrong ty detection

* fix link

* ensure tracing does not enagage while running backup ops

* fix import

* remove debugging string

* add error codes

* add tests for backup and restore

* complete test for restore

* we do not need e2e on every matrix entry
there is no realy db dep here

* fix changelog format

* add flag to allow bypass
This commit is contained in:
Matthias Mair
2026-02-25 00:23:00 +01:00
committed by GitHub
parent 8a4ad4ff62
commit ac9a1f2251
7 changed files with 329 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
CHANGELOG.md
View File

@@ -13,13 +13,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Added
[#11405](https://github.com/inventree/InvenTree/pull/11405) adds default table filters, which hide inactive items by default. The default table filters are overridden by user filter selection, and only apply to the table view initially presented to the user. This means that users can still view inactive items if they choose to, but they will not be shown by default.
[#11222](https://github.com/inventree/InvenTree/pull/11222) adds support for data import using natural keys, allowing for easier association of related objects without needing to know their internal database IDs.
[#11383](https://github.com/inventree/InvenTree/pull/11383) adds "exists_for_model_id", "exists_for_related_model", and "exists_for_related_model_id" filters to the ParameterTemplate API endpoint. These filters allow users to check for the existence of parameters associated with specific models or related models, improving the flexibility and usability of the API.
[#10887](https://github.com/inventree/InvenTree/pull/10887) adds the ability to auto-allocate tracked items against specific build outputs. Currently, this will only allocate items where the serial number of the tracked item matches the serial number of the build output, but in future this may be extended to allow for more flexible allocation rules.
- [#11405](https://github.com/inventree/InvenTree/pull/11405) adds default table filters, which hide inactive items by default. The default table filters are overridden by user filter selection, and only apply to the table view initially presented to the user. This means that users can still view inactive items if they choose to, but they will not be shown by default.
- [#11222](https://github.com/inventree/InvenTree/pull/11222) adds support for data import using natural keys, allowing for easier association of related objects without needing to know their internal database IDs.
- [#11383](https://github.com/inventree/InvenTree/pull/11383) adds "exists_for_model_id", "exists_for_related_model", and "exists_for_related_model_id" filters to the ParameterTemplate API endpoint. These filters allow users to check for the existence of parameters associated with specific models or related models, improving the flexibility and usability of the API.
- [#10887](https://github.com/inventree/InvenTree/pull/10887) adds the ability to auto-allocate tracked items against specific build outputs. Currently, this will only allocate items where the serial number of the tracked item matches the serial number of the build output, but in future this may be extended to allow for more flexible allocation rules.
- [#11372](https://github.com/inventree/InvenTree/pull/11372) adds backup metadata setter and restore metadata validator functions to ensure common footguns are harder to trigger when using the backup and restore functionality.
### Changed

22
docs/docs/settings/error_codes.md
View File

@@ -84,23 +84,27 @@ This is a security measure to prevent plugins from changing the core functionali
An error occurred when discovering or initializing a machine type from a plugin. This likely indicates a faulty or incompatible plugin.
#### INVE-E13
**Error reading InvenTree configuration file**
An error occurred while reading the InvenTree configuration file. This might be caused by a syntax error or invalid value in the configuration file.
#### INVE-E14
**Could not import Django**
Django is not installed in the current Python environment. This means that the InvenTree backend is not running within the correct [python virtual environment](../start/index.md#virtual-environment) or that the required Python packages were not installed correctly.
#### INVE-E15
**Python version not supported**
This error occurs attempting to run InvenTree on a version of Python which is older than the minimum required version. We [require Python {{ config.extra.min_python_version }} or newer](../start/index.md#python-requirements)
#### INVE-E16
**Restore was stopped due to critical issues with backup environment - Backend**
A potentially critical mismatch between the backup environment and the current restore environment was detected during the restore process. The restore was stopped to prevent potential data loss or corruption. Check the logs for more information about the detected issues.
While using [invoke](../start/invoke.md), this can be overridden with the `--restore-allow-newer-version` flag.
### INVE-W (InvenTree Warning)
Warnings - These are non-critical errors which should be addressed when possible.
@@ -199,6 +203,14 @@ A user attempted to sign up but registration is currently disabled via the syste
To enable registration, adjust the relevant settings (for regular or SSO registration) to allow user signups.
#### INVE-W13
**Current environment inconsistent with backup environment - Backend**
The environment in which the backup was taken does not match the current environment. This might lead to issues with restoring the backup, as the backup might contain data that is not compatible with the current environment. Plugins for example might be missing or are present in a different version - this can lead to issues with restoring the backup.
This warning will not prevent you from restoring the backup but it is recommended to ensure the mentioned issues are resolved before restoring the backup to prevent issues with the restored instance.
### INVE-I (InvenTree Information)
Information — These are not errors but information messages. They might point out potential issues or just provide information.
@@ -215,5 +227,9 @@ An issue was detected with the application of a filtering serializer or decorato
This warning should only be raised during development and not in production, if you recently installed a plugin you might want to contact the plugin author.
#### INVE-I3
**Backup and restore information - Backend**
Information about the metadata of a backup being restored or the environment in which a backup is being restored to. This information can be helpful to understand the backup and restore process and to identify potential issues with the backup or the restore process.
### INVE-M (InvenTree Miscellaneous)
Miscellaneous — These are information messages that might be used to mark debug information or other messages helpful for the InvenTree team to understand behaviour.

1
docs/docs/start/backup.md
View File

@@ -29,6 +29,7 @@ The following configuration options are available for backup:
| INVENTREE_BACKUP_DATE_FORMAT | backup_date_format | Date format string used to format timestamps in backup filenames. | `%Y-%m-%d-%H%M%S` |
| INVENTREE_BACKUP_DATABASE_FILENAME_TEMPLATE | backup_database_filename_template | Template string used to generate database backup filenames. | `InvenTree-db-{datetime}.{extension}` |
| INVENTREE_BACKUP_MEDIA_FILENAME_TEMPLATE | backup_media_filename_template | Template string used to generate media backup filenames. | `InvenTree-media-{datetime}.{extension}` |
| INVENTREE_BACKUP_RESTORE_ALLOW_NEWER_VERSION | backup_restore_allow_newer_version | If True, allows restoring a backup created with a newer version of InvenTree. This is dangerous as it can lead to hard-to-debug data loss. | False |
### Storage Backend

137
src/backend/InvenTree/InvenTree/backup.py
View File

@@ -5,7 +5,16 @@ We use the django-dbbackup library to handle backup and restore operations.
Ref: https://archmonger.github.io/django-dbbackup/latest/configuration/
"""
from datetime import datetime, timedelta
from django.conf import settings
import structlog
import InvenTree.config
import InvenTree.version
logger = structlog.get_logger('inventree')
def get_backup_connector_options() -> dict:
@@ -131,3 +140,131 @@ def backup_media_filename_template() -> str:
default_value='InvenTree-media-{datetime}.{extension}',
typecast=str,
)
# schema for backup metadata
InvenTreeBackupMetadata = dict[str, str | int | bool | None]
def _gather_environment_metadata() -> InvenTreeBackupMetadata:
"""Gather metadata about the current environment to be stored with the backup."""
import plugin.installer
new_data: InvenTreeBackupMetadata = {}
new_data['ivt_1_debug'] = settings.DEBUG
new_data['ivt_1_version'] = InvenTree.version.inventreeVersion()
new_data['ivt_1_version_api'] = InvenTree.version.inventreeApiVersion()
new_data['ivt_1_plugins_enabled'] = settings.PLUGINS_ENABLED
new_data['ivt_1_plugins_file_hash'] = plugin.installer.plugins_file_hash()
new_data['ivt_1_installer'] = InvenTree.config.inventreeInstaller()
new_data['ivt_1_backup_time'] = datetime.now().isoformat()
return new_data
def _parse_environment_metadata(metadata: InvenTreeBackupMetadata) -> dict[str, str]:
"""Parse backup metadata to extract environment information."""
data = {}
data['debug'] = metadata.get('ivt_1_debug', False)
data['version'] = metadata.get('ivt_1_version', 'unknown')
data['version_api'] = metadata.get('ivt_1_version_api', 'unknown')
data['plugins_enabled'] = metadata.get('ivt_1_plugins_enabled', False)
data['plugins_file_hash'] = metadata.get('ivt_1_plugins_file_hash', 'unknown')
data['installer'] = metadata.get('ivt_1_installer', 'unknown')
data['backup_time'] = metadata.get('ivt_1_backup_time', 'unknown')
return data
def metadata_set(metadata) -> InvenTreeBackupMetadata:
"""Set backup metadata for the current backup operation."""
return _gather_environment_metadata()
def validate_restore(metadata: InvenTreeBackupMetadata) -> bool | None:
"""Validate whether a backup restore operation should proceed, based on the provided metadata."""
if metadata.get('ivt_1_version') is None:
logger.warning(
'INVE-W13: Backup metadata does not contain version information',
error_code='INVE-W13',
)
return True
current_environment = _parse_environment_metadata(_gather_environment_metadata())
backup_environment = _parse_environment_metadata(metadata)
# Version mismatch
if backup_environment['version'] != current_environment['version']:
logger.warning(
f'INVE-W13: Backup being restored was created with InvenTree version {backup_environment["version"]}, but current version is {current_environment["version"]}',
error_code='INVE-W13',
)
# Backup is from newer version - fail
try:
if int(backup_environment['version_api']) > int(
str(current_environment['version_api'])
):
logger.error(
'INVE-E16: Backup being restored was created with a newer version of InvenTree - restore cannot proceed. If you are using the invoke task for your restore this warning might be overridden once with `--restore-allow-newer-version`',
error_code='INVE-E16',
)
# Check for pass flag to allow restore
if not settings.BACKUP_RESTORE_ALLOW_NEWER_VERSION: # defaults to False
return False
else:
logger.warning(
'INVE-W13: Backup restore is allowing a restore from a newer version of InvenTree - this can lead to data loss or corruption',
error_code='INVE-W13',
)
except ValueError: # pragma: no cover
logger.warning(
'INVE-W13: Could not parse API version from backup metadata - cannot determine if backup is from newer version',
error_code='INVE-W13',
)
# Plugins enabled on backup but not restore environment - warn
if (
backup_environment['plugins_enabled']
and not current_environment['plugins_enabled']
):
logger.warning(
'INVE-W13: Backup being restored was created with plugins enabled, but current environment has plugins disabled - this can lead to data loss',
error_code='INVE-W13',
)
# Plugins file hash mismatch - warn
if pg_hash := backup_environment['plugins_file_hash']:
if pg_hash != current_environment['plugins_file_hash']:
logger.warning(
'INVE-W13: Backup being restored has a different plugins file hash to the current environment - this can lead to data loss or corruption',
error_code='INVE-W13',
)
# Installer mismatch - warn
if installer := backup_environment['installer']:
if installer != current_environment['installer']:
logger.warning(
f"INVE-W13: Backup being restored was created with installer '{installer}', but current environment has installer '{current_environment['installer']}'",
error_code='INVE-W13',
)
# Age of backup
last_backup_time = backup_environment.get('backup_time')
if datetime.now() - datetime.fromisoformat(last_backup_time) > timedelta(days=120):
logger.warning(
f'INVE-W13: Backup being restored is over 120 days old (last backup time: {last_backup_time})',
error_code='INVE-W13',
)
if settings.DEBUG: # pragma: no cover
logger.info(
f'INVE-I3: Backup environment: {backup_environment}', error_code='INVE-I3'
)
logger.info(
f'INVE-I3: Current environment: {current_environment}', error_code='INVE-I3'
)
return True

14
src/backend/InvenTree/InvenTree/settings.py
View File

@@ -27,7 +27,7 @@ from corsheaders.defaults import default_headers as default_cors_headers
import InvenTree.backup
from InvenTree.cache import get_cache_config, is_global_cache_enabled
from InvenTree.config import get_boolean_setting, get_oidc_private_key, get_setting
from InvenTree.ready import isInMainThread
from InvenTree.ready import isInMainThread, isRunningBackup
from InvenTree.sentry import default_sentry_dsn, init_sentry
from InvenTree.version import checkMinPythonVersion, inventreeCommitHash
from users.oauth2_scopes import oauth2_scopes
@@ -258,12 +258,22 @@ DBBACKUP_EMAIL_SUBJECT_PREFIX = InvenTree.backup.backup_email_prefix()
DBBACKUP_CONNECTORS = {'default': InvenTree.backup.get_backup_connector_options()}
DBBACKUP_BACKUP_METADATA_SETTER = InvenTree.backup.metadata_set
DBBACKUP_RESTORE_METADATA_VALIDATOR = InvenTree.backup.validate_restore
# Data storage options
DBBACKUP_STORAGE_CONFIG = {
'BACKEND': InvenTree.backup.get_backup_storage_backend(),
'OPTIONS': InvenTree.backup.get_backup_storage_options(),
}
# This can also be overridden with a command line flag --restore-allow-newer-version when running the restore command
BACKUP_RESTORE_ALLOW_NEWER_VERSION = get_boolean_setting(
'INVENTREE_BACKUP_RESTORE_ALLOW_NEWER_VERSION',
'backup_restore_allow_newer_version',
False,
)
# Enable django admin interface?
INVENTREE_ADMIN_ENABLED = get_boolean_setting(
'INVENTREE_ADMIN_ENABLED', config_key='admin_enabled', default_value=True
@@ -862,7 +872,7 @@ TRACING_ENABLED = get_boolean_setting(
)
TRACING_DETAILS: Optional[dict] = None
if TRACING_ENABLED: # pragma: no cover
if TRACING_ENABLED and not isRunningBackup(): # pragma: no cover
from InvenTree.tracing import setup_instruments, setup_tracing
_t_endpoint = get_setting('INVENTREE_TRACING_ENDPOINT', 'tracing.endpoint', None)

146
src/backend/InvenTree/InvenTree/test_commands.py
View File

@@ -2,10 +2,15 @@
from pathlib import Path
from django.conf import settings
from django.contrib.auth.models import User
from django.core.management import call_command
from django.test import TestCase
from opentelemetry.instrumentation.sqlite3 import SQLite3Instrumentor
from InvenTree.config import get_testfolder_dir
class CommandTestCase(TestCase):
"""Test case for custom management commands."""
@@ -66,3 +71,144 @@ class CommandTestCase(TestCase):
output = call_command('remove_mfa', username=my_admin3.username, verbosity=0)
self.assertEqual(output, 'done')
self.assertEqual(my_admin3.authenticator_set.all().count(), 0)
def test_backup_metadata(self):
"""Test the backup metadata functions."""
from InvenTree.backup import (
_gather_environment_metadata,
_parse_environment_metadata,
)
metadata = _gather_environment_metadata()
self.assertIn('ivt_1_version', metadata)
self.assertIn('ivt_1_plugins_enabled', metadata)
parsed = _parse_environment_metadata(metadata)
self.assertIn('version', parsed)
self.assertIn('plugins_enabled', parsed)
def test_restore_validation(self):
"""Test the restore validation functions."""
from InvenTree.backup import _gather_environment_metadata, validate_restore
metadata = _gather_environment_metadata()
self.assertTrue(validate_restore(metadata))
# Version
with self.assertLogs() as cm:
self.assertTrue(validate_restore({}))
self.assertIn(
'INVE-W13: Backup metadata does not contain version information', str(cm[1])
)
with self.assertLogs() as cm:
self.assertTrue(validate_restore({**metadata, 'ivt_1_version': '123xx'}))
self.assertIn(
'INVE-W13: Backup being restored was created with InvenTree version',
str(cm[1]),
)
with self.assertLogs() as cm:
self.assertFalse(
validate_restore({
**metadata,
'ivt_1_version': '9999',
'ivt_1_version_api': '9999',
})
)
self.assertIn(
'INVE-E16: Backup being restored was created with a newer version',
str(cm[1]),
)
# not with allow flag
with self.settings(BACKUP_RESTORE_ALLOW_NEWER_VERSION=True):
with self.assertLogs() as cm:
self.assertTrue(
validate_restore({
**metadata,
'ivt_1_version': '9999',
'ivt_1_version_api': '9999',
})
)
self.assertIn(
'INVE-W13: Backup restore is allowing a restore from a newer version of InvenTree',
str(cm[1]),
)
# Plugins enabled
with self.settings(PLUGINS_ENABLED=False):
with self.assertLogs() as cm:
self.assertTrue(validate_restore(metadata))
self.assertIn(
'INVE-W13: Backup being restored was created with plugins enabled',
str(cm[1]),
)
# Plugin hash
with self.assertLogs() as cm:
self.assertTrue(
validate_restore({**metadata, 'ivt_1_plugins_file_hash': '123xx'})
)
self.assertIn(
'INVE-W13: Backup being restored has a different plugins file hash',
str(cm[1]),
)
# installer
with self.assertLogs() as cm:
self.assertTrue(validate_restore({**metadata, 'ivt_1_installer': '123xx'}))
self.assertIn(
'INVE-W13: Backup being restored was created with installer', str(cm[1])
)
# Age
with self.assertLogs() as cm:
self.assertTrue(
validate_restore({
**metadata,
'ivt_1_backup_time': '2020-02-02T00:00:00',
})
)
self.assertIn(
'INVE-W13: Backup being restored is over 120 days old', str(cm[1])
)
def test_backup_command_e2e(self):
"""Test the backup command."""
# we only test on sqlite, the environment in which we also run coverage
if settings.DB_ENGINE != 'django.db.backends.sqlite3':
self.skipTest('Backup command test only runs on sqlite database')
# disable tracing for now
if settings.TRACING_ENABLED: # pragma: no cover
print('Disabling tracing for backup command test')
SQLite3Instrumentor().uninstrument()
output_path = get_testfolder_dir().joinpath('backup.zip').resolve()
# Backup
with self.assertLogs() as cm:
output = call_command(
'dbbackup', noinput=True, verbosity=2, output_path=str(output_path)
)
self.assertIsNone(output)
self.assertIn(f'Writing metadata file to {output_path}', str(cm[1]))
# Restore
with self.assertLogs() as cm:
output = call_command(
'dbrestore',
noinput=True,
interactive=False,
verbosity=2,
input_path=str(output_path),
)
self.assertIsNone(output)
self.assertIn('Using connector from metadata', str(cm[1]))
# Cleanup the generated backup file and metadata file
output_path.unlink()
Path(str(output_path) + '.metadata').unlink()
if settings.TRACING_ENABLED: # pragma: no cover
print('Re-enabling tracing for backup command test')
SQLite3Instrumentor().instrument()

11
tasks.py
View File

@@ -793,6 +793,7 @@ def backup(
'skip_db': 'Do not import database archive (media restore only)',
'skip_media': 'Do not import media archive (database restore only)',
'uncompress': 'Uncompress the backup files before restoring (default behavior)',
'restore_allow_newer_version': 'Allow restore from a newer version backup (use with caution)',
}
)
def restore(
@@ -804,10 +805,16 @@ def restore(
skip_db: bool = False,
skip_media: bool = False,
uncompress: bool = True,
restore_allow_newer_version: bool = False,
):
"""Restore the database and media files."""
base_cmd = '--noinput -v 2'
env = {}
if restore_allow_newer_version:
env['INVENTREE_BACKUP_RESTORE_ALLOW_NEWER_VERSION'] = 'True'
if uncompress:
base_cmd += ' --uncompress'
@@ -831,7 +838,7 @@ def restore(
if db_file:
cmd += f' -i {db_file}'
manage(c, cmd)
manage(c, cmd, env=env)
if skip_media:
info('Skipping media restore...')
@@ -842,7 +849,7 @@ def restore(
if media_file:
cmd += f' -i {media_file}'
manage(c, cmd)
manage(c, cmd, env=env)
@task()