diff --git a/CHANGELOG.md b/CHANGELOG.md index 8db8b4469e..7b6b2c9523 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Breaking Changes - [#13034](https://github.com/inventree/InvenTree/pull/13034) moves pricing models into a new dedicated "pricing" model, and provides new API endpoints for managing pricing separately from other models. +- [#13101](https://github.com/inventree/InvenTree/pull/13101) removes the StockTracking model and its associated API endpoints. Stock history tracking is now handled by the global audit log. ### Added @@ -19,6 +20,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Removed +- [#13101](https://github.com/inventree/InvenTree/pull/13101) removes the StockTracking model and its associated API endpoints. Stock history tracking is now handled by the global audit log. + ## 1.6.0 - 2026-10-18 ### Breaking Changes diff --git a/docs/docs/assets/images/stock/part_stock_history.png b/docs/docs/assets/images/stock/part_stock_history.png new file mode 100644 index 0000000000..74456046c4 Binary files /dev/null and b/docs/docs/assets/images/stock/part_stock_history.png differ diff --git a/docs/docs/assets/images/stock/part_tracking_history.png b/docs/docs/assets/images/stock/part_tracking_history.png deleted file mode 100644 index cf31097ffe..0000000000 Binary files a/docs/docs/assets/images/stock/part_tracking_history.png and /dev/null differ diff --git a/docs/docs/assets/images/stock/stock_item_history.png b/docs/docs/assets/images/stock/stock_item_history.png new file mode 100644 index 0000000000..3ec4336a54 Binary files /dev/null and b/docs/docs/assets/images/stock/stock_item_history.png differ diff --git a/docs/docs/assets/images/stock/stock_item_tracking_history.png b/docs/docs/assets/images/stock/stock_item_tracking_history.png deleted file mode 100644 index 895fb61e04..0000000000 Binary files a/docs/docs/assets/images/stock/stock_item_tracking_history.png and /dev/null differ diff --git a/docs/docs/concepts/audit.md b/docs/docs/concepts/audit.md index 1a8f409237..a11b5d6231 100644 --- a/docs/docs/concepts/audit.md +++ b/docs/docs/concepts/audit.md @@ -36,7 +36,6 @@ To keep the audit log focused and performant, some changes are intentionally *no - **Derived fields** - Internal or calculated fields (such as tree structure fields, cached pricing values and metadata) are excluded. - **Data import** - Changes made using the [data import](./data_import.md) tool are not recorded. -- **Stock items** - Changes to stock items are tracked separately using [stock tracking](../stock/tracking.md). - **Unsupported models** - Only the models listed below are audited. ### Audited Models @@ -45,6 +44,21 @@ The following models are currently audited. Where a model has a *parent*, change {{ audited_models() }} +### Events + +Some models record named *events* in the audit log, in addition to field changes. An event describes *what happened* to an item (for example, an operation which affects multiple fields, or multiple items), and may include: + +- **Event type** - The type of event (displayed in place of the action in the audit log table) +- **Notes** - Optional notes provided by the user when the event was recorded +- **Deltas** - Additional information about the event, such as quantities or references to related objects. References to related objects are recorded as they were at the time of the event. + +An event may be recorded even if no fields of the item were changed. + +For example, stock operations (such as moving, counting or shipping stock) are recorded as [stock tracking](../stock/tracking.md) events against each affected stock item. + +!!! warning "Disabling the Audit Log" + Events are recorded in the audit log, and are subject to the same settings as all other audit entries. If the audit log is disabled, events are **not** recorded - this includes **all stock tracking history**. + ### Viewing Change History #### Detail Pages @@ -60,9 +74,10 @@ For example, the history panel for a purchase order shows changes made to the or The *History* panel is only displayed if **all** of the following conditions are met: - The audit log is enabled (via the `Enable Audit Log` system setting) -- The user is a *staff* user - The user has enabled the *Show Change History* user setting +The entries displayed in the *History* panel are filtered by the permissions of the user (see [Permissions](#permissions) below). + {{ image("concepts/audit_user_setting.png", "Show change history setting") }} #### Admin Center @@ -79,10 +94,12 @@ The audit log table can be filtered by action, user, date range and model type: Access to the audit log is restricted: -- The audit log (including the `/api/audit/` API endpoint) is only available to *staff* users. -- Audit entries are additionally filtered by the user's [role permissions](../settings/permissions.md). A user can only view audit entries for models which they have permission to view. +- Audit entries are filtered by the user's [role permissions](../settings/permissions.md). A user can only view audit entries for models which they have permission to view. - Some models (e.g. attachments and parameters) require the user to also have permission to view the *parent* model. +- The history of some models (e.g. global settings) is only available to *staff* users. - The remote IP address associated with each entry is only visible to staff users. +- References to related objects (recorded against [events](#events)) are hidden if the user does not have permission to view the related model. +- The *Audit Log* panel in the Admin Center is only available to staff users. ### Data Retention diff --git a/docs/docs/manufacturing/output.md b/docs/docs/manufacturing/output.md index 2f98c2ff16..886dd3747e 100644 --- a/docs/docs/manufacturing/output.md +++ b/docs/docs/manufacturing/output.md @@ -41,7 +41,7 @@ The following options are available when creating a new build output: ### Specifying Serial Numbers -Refer to the [serial number generation guide](../stock/tracking.md#generating-serial-numbers) for further information on serial number input. +Refer to the [serial number generation guide](../stock/traceability.md#generating-serial-numbers) for further information on serial number input. ### Creating Outputs Without Serial Numbers diff --git a/docs/docs/plugins/mixins/validation.md b/docs/docs/plugins/mixins/validation.md index bbdbc2207f..8dbf917928 100644 --- a/docs/docs/plugins/mixins/validation.md +++ b/docs/docs/plugins/mixins/validation.md @@ -150,7 +150,7 @@ Validation of the Part IPN (Internal Part Number) field is exposed to custom plu ### Batch Codes -[Batch codes](../../stock/tracking.md#batch-codes) can be generated and/or validated by custom plugins. +[Batch codes](../../stock/traceability.md#batch-codes) can be generated and/or validated by custom plugins. #### Validate Batch Code @@ -184,7 +184,7 @@ The `generate_batch_code` method can be implemented to generate a new batch code Requirements for serial numbers can vary greatly depending on the application. Rather than attempting to provide a "one size fits all" serial number implementation, InvenTree allows custom serial number schemes to be implemented via plugins. -The default InvenTree [serial numbering system](../../stock/tracking.md#serial-numbers) uses a simple algorithm to validate and increment serial numbers. More complex behaviors can be implemented using the `ValidationMixin` plugin class and the following custom methods: +The default InvenTree [serial numbering system](../../stock/traceability.md#serial-numbers) uses a simple algorithm to validate and increment serial numbers. More complex behaviors can be implemented using the `ValidationMixin` plugin class and the following custom methods: #### Serial Number Validation diff --git a/docs/docs/sales/return_order.md b/docs/docs/sales/return_order.md index 79db9c1420..8e0b1ba9f9 100644 --- a/docs/docs/sales/return_order.md +++ b/docs/docs/sales/return_order.md @@ -145,7 +145,7 @@ Receiving a line item performs the following actions on the underlying stock ite * Sets the stock item status (defaulting to *Quarantined*) * Removes the *Customer* reference from the stock item * Clears any outstanding sales order allocations against the stock item -* Adds a tracking entry recording the return +* Records a [stock tracking](../stock/tracking.md) event for the return !!! info "Partial Returns" If a customer returns less than the full quantity of a non-serialized stock item, InvenTree automatically splits the returned quantity into a new stock item, leaving the remainder of the original stock item untouched (e.g. still recorded as sold to the customer). diff --git a/docs/docs/stock/adjust.md b/docs/docs/stock/adjust.md index c536bf0670..2c75360798 100644 --- a/docs/docs/stock/adjust.md +++ b/docs/docs/stock/adjust.md @@ -12,7 +12,7 @@ Stock adjustments can be accessed in any stock items table using the "Stock Opti ### Move Stock -Multiple stock items can be moved to a new location in a single operation. Each item is moved to the selected location, and a stock tracking entry is added to the stock item history. +Multiple stock items can be moved to a new location in a single operation. Each item is moved to the selected location, and a [stock tracking](./tracking.md) event is recorded against each stock item. {{ image("stock/stock_move.png", "Stock Move") }} diff --git a/docs/docs/stock/disassemble.md b/docs/docs/stock/disassemble.md index dd03877c35..2c5aa07016 100644 --- a/docs/docs/stock/disassemble.md +++ b/docs/docs/stock/disassemble.md @@ -44,7 +44,7 @@ The original stock item is never deleted as a result of disassembly - its quanti If the stock item being disassembled has other stock items [installed](../manufacturing/allocate.md#allocating-tracked-stock) within it (for example, tracked components that were installed during a build order), these installed items **must** be accounted for during disassembly: - Each installed item is matched against a BOM line item, based on the component part (including any [substitute](../manufacturing/bom.md#substitute-bom-line-items) or [variant](../part/index.md#assembly) parts allowed for that line) -- Matched installed items are *uninstalled* directly, rather than being discarded and re-created - this preserves the original stock item, including its own tracking history, batch code, and purchase price +- Matched installed items are *uninstalled* directly, rather than being discarded and re-created - this preserves the original stock item, including its own stock tracking history, batch code, and purchase price - The quantity requested for the matching BOM line is reduced by the quantity already covered by the installed item(s). A new stock item is only created for any remaining quantity - if the installed items fully cover the required quantity, no new stock item is created for that line - Any installed item which does *not* match one of the selected BOM lines is still uninstalled (it cannot be left "installed" inside a smaller or non-existent parent), but its quantity is **not** subtracted from any line @@ -65,10 +65,10 @@ If the original stock item has a recorded purchase price, and no explicit *Unit Disassembling a stock item generates a full audit trail: -- A `Disassembled into components` entry is added to the tracking history of the original stock item -- Each newly created component stock item receives a `Created from disassembly` tracking entry, referencing the original stock item +- A `Disassembled into components` event is added to the [stock tracking history](./tracking.md) of the original stock item +- Each newly created component stock item records a `Created from disassembly` event, referencing the original stock item - The *batch code* and source *purchase order* of the original stock item are copied directly to each generated component -- If the original stock item was generated by a build order, that build order cannot be directly copied to the new component (since the component was not actually built by that order) - instead, it is recorded as a reference within the `Created from disassembly` tracking entry +- If the original stock item was generated by a build order, that build order cannot be directly copied to the new component (since the component was not actually built by that order) - instead, it is recorded as a reference within the `Created from disassembly` event ### Purchasing Bundled Items diff --git a/docs/docs/stock/index.md b/docs/docs/stock/index.md index 83ec1e5f78..12916fd9fd 100644 --- a/docs/docs/stock/index.md +++ b/docs/docs/stock/index.md @@ -44,9 +44,9 @@ Refer to the [traceability](./traceability.md) page for more information on how ## Stock Tracking -Every time a *Stock Item* is adjusted, a *Stock Tracking* entry is automatically created. This ensures a complete history of the *Stock Item* is maintained as long as the item is in the system. +Every time a *Stock Item* is adjusted, a stock tracking event is automatically recorded in the [audit log](../concepts/audit.md). This ensures a complete history of the *Stock Item* is maintained. -Each stock tracking historical item records the user who performed the action. [Read more about stock tracking here](./tracking.md). +Each stock tracking event records the user who performed the action. [Read more about stock tracking here](./tracking.md). ## Stock Locations diff --git a/docs/docs/stock/tracking.md b/docs/docs/stock/tracking.md index 12ac65c48f..d412f0bc1f 100644 --- a/docs/docs/stock/tracking.md +++ b/docs/docs/stock/tracking.md @@ -4,47 +4,56 @@ title: Stock Tracking ## Stock Tracking -Stock tracking entries record the history of stock item adjustments, including the user who performed the action, the date of the action, and the quantity change. This allows users to maintain a complete history of stock item movements and adjustments over time. +Stock tracking records the history of stock item adjustments, including the user who performed the action, the date of the action, and the changes made to the stock item (e.g. quantity or location). This allows users to maintain a complete history of stock item movements and adjustments over time. + +Stock tracking history is recorded in the [audit log](../concepts/audit.md) - each stock operation is recorded as a named [event](../concepts/audit.md#events) against each affected stock item. ### Tracking Events -Stock tracking entries are created automatically whenever a stock item is adjusted, either through manual adjustments or through automated processes such as order fulfillment or build completion. +Stock tracking events are recorded automatically whenever a stock item is adjusted, either through manual adjustments or through automated processes such as order fulfillment or build completion. -Some examples of events that may trigger stock tracking entries include: +Some examples of operations that record stock tracking events include: - Manual stock adjustments (e.g. correcting inventory counts) - Creation of new stock items (e.g. receiving new inventory) - Allocation of stock items to orders (e.g. shipping items against sales orders) - Consumption of stock items during build processes (e.g. using items to complete a build order) -### Tracking Entry Types +### Tracking Event Types -Each stock tracking entry records a specific *type*, indicating which event triggered it: +Each stock tracking event records a specific *type*, indicating which operation triggered it: {{ statuscodes("StockHistoryCode") }} +!!! info "Legacy Event Types" + The *Legacy stock tracking entry*, *Edited stock item* and *Assigned serial number* event types are only present in stock history recorded by older versions of InvenTree. Edits to a stock item (e.g. changing the status) are now recorded as regular updates in the audit log, and serial numbers are recorded as part of the creation of the stock item. + ## Viewing Stock Tracking History -There are multiple ways to view the stock tracking history for a particular stock item or part via the user interface. +Stock tracking events are recorded in the [audit log](../concepts/audit.md), along with any other changes made to a stock item. Each event records the user who performed the action, any notes provided, and additional information about the event (such as the quantity added or removed, or the order which the item was received against). -### Stock Item Tracking History +!!! warning "Audit Log Required" + Stock tracking history is recorded in the audit log. If the audit log is disabled (via the `Enable Audit Log` system setting), **no stock tracking history is recorded**. Refer to the [audit log documentation](../concepts/audit.md#settings) for more information. -The stock tracking history for a particular stock item can be viewed on the *Stock Item Detail* page, under the *Stock Tracking* tab: +### Stock Item History -{{ image("stock/stock_item_tracking_history.png", title="Stock tracking tab") }} +The stock tracking history for a particular stock item is displayed in the *History* panel of the *Stock Item Detail* page: -This view displays all tracking entries associated with the particular stock item. +{{ image("stock/stock_item_history.png", "Stock item history") }} -### Part Tracking History +Each entry displays the type of event, along with any changes made to the stock item, and any additional information recorded against the event (such as notes, quantities, or links to related orders and stock items). -Additionally, the stock tracking history for a particular part can be viewed on the *Part Detail* page, under the *Stock History* tab: +### Part History -{{ image("stock/part_tracking_history.png", title="Part stock tracking history") }} +The history of all stock items for a particular part is displayed in the *History* panel of the *Part Detail* page, along with any changes made to the part itself: -This view displays all tracking entries associated with any stock item linked to the particular part. +{{ image("stock/part_stock_history.png", "Part stock history") }} + +!!! info "Show Change History" + The *History* panel is only displayed if the *Show Change History* user setting is enabled. Refer to the [audit log documentation](../concepts/audit.md#viewing-change-history) for more information. !!! info "Deleted Stock Items" - Even if a stock item is deleted from the system, the associated stock tracking entries are retained for historical reference. They will be visible in the part tracking history, but not in the stock item tracking history (as the stock item itself has been deleted). + Even if a stock item is deleted from the system, the associated history is retained for historical reference. It is visible in the history of the part, but not in the history of the stock item (as the stock item itself has been deleted). ## Installed Stock Items @@ -73,9 +82,7 @@ Additionally, installed items are automatically uninstalled when the parent item ## Stock Tracking Settings -There are a number of configuration options available for controlling the behavior of stock tracking functionality in the [system settings view](../settings/global.md): +Stock tracking history is recorded in the audit log, and is subject to the [audit log settings](../concepts/audit.md#settings). For example, old stock tracking history can be automatically removed using the {{ globalsetting("AUDIT_LOG_DELETE_OLD_ENTRIES", short=True) }} setting. -| Name | Description | Default | Units | -| ---- | ----------- | ------- | ----- | -{{ globalsetting("STOCK_TRACKING_DELETE_OLD_ENTRIES") }} -{{ globalsetting("STOCK_TRACKING_DELETE_DAYS") }} +!!! info "Audit Log Retention" + The *Delete Old Stock Tracking Entries* and *Stock Tracking Deletion Interval* settings have been removed. The retention of stock tracking history is now controlled by the audit log settings. diff --git a/src/backend/InvenTree/InvenTree/api_version.py b/src/backend/InvenTree/InvenTree/api_version.py index be5ad33454..d2a6740b80 100644 --- a/src/backend/InvenTree/InvenTree/api_version.py +++ b/src/backend/InvenTree/InvenTree/api_version.py @@ -1,11 +1,20 @@ """InvenTree API version information.""" # InvenTree API version -INVENTREE_API_VERSION = 565 +INVENTREE_API_VERSION = 566 """Increment this API version number whenever there is a significant change to the API that any clients need to know about.""" INVENTREE_API_TEXT = """ +v566 -> 2026-10-25 : https://github.com/inventree/InvenTree/pull/13101 + - Audit log API (/api/audit/) is available to all authenticated users (entries are filtered by model permissions) + - Adds "event_type", "event_label", "notes" and "deltas" fields to the audit log API + - Adds "event_type" filter to the audit log API + - Stock item history is recorded in the audit log (StockItem events, with event_type values from StockHistoryCode) + - Breaking: Removes the stock tracking API endpoints (/api/stock/track/, /api/stock/track//, /api/stock/track/status/) - use /api/audit/ instead + - Breaking: Removes the "tracking_items" field from the StockItem API + - Breaking: Removes the "copy_history" option when duplicating a StockItem + v565 -> 2026-10-24 : https://github.com/inventree/InvenTree/pull/13069 - Adds read-only audit log API endpoints (/api/audit/) - OPTIONS metadata includes a "filters" entry, listing the available choices for choice-based list filters diff --git a/src/backend/InvenTree/InvenTree/apps.py b/src/backend/InvenTree/InvenTree/apps.py index 1a7687d0e4..10560122ce 100644 --- a/src/backend/InvenTree/InvenTree/apps.py +++ b/src/backend/InvenTree/InvenTree/apps.py @@ -117,6 +117,7 @@ class InvenTreeConfig(AppConfig): 'label.tasks.cleanup_old_label_outputs', 'report.tasks.cleanup_old_report_outputs', 'stock.tasks.delete_old_stock_items', + 'stock.tasks.delete_old_stock_tracking', ] try: diff --git a/src/backend/InvenTree/InvenTree/management/commands/bulkloaddata.py b/src/backend/InvenTree/InvenTree/management/commands/bulkloaddata.py index d5e8f43274..8bd4ded893 100644 --- a/src/backend/InvenTree/InvenTree/management/commands/bulkloaddata.py +++ b/src/backend/InvenTree/InvenTree/management/commands/bulkloaddata.py @@ -31,7 +31,7 @@ def _suspend_auto_now(model): use the fixture's own value. bulk_create() has no equivalent of that raw=True path - it always calls Field.pre_save(), which unconditionally stamps such fields with timezone.now(), discarding whatever value the - fixture provided (e.g. stock.StockItemTracking.date). Clearing the flags + fixture provided (e.g. stock.StockItem.creation_date). Clearing the flags for the duration of the bulk_create() call restores the raw=True behaviour for this model. """ @@ -65,7 +65,7 @@ class Command(LoadDataCommand): auto_now / auto_now_add fields are a partial exception: they are temporarily disabled around the bulk_create() call (see _suspend_auto_now()) so that fixture-provided values for fields such as - stock.StockItemTracking.date survive the import, matching what a + stock.StockItem.creation_date survive the import, matching what a normal, non-bulk fixture load does via raw=True. - Multi-table inheritance is not supported by bulk_create() and will fail loudly rather than being silently mishandled. Natural-key foreign key / @@ -74,7 +74,7 @@ class Command(LoadDataCommand): sped up by caching each resolved natural key for the life of the command (see _cached_natural_keys()), since 'export_records' uses --natural-foreign and a large fixture can have many rows referencing the - same handful of natural-keyed objects (e.g. stock.StockItemTracking.user + same handful of natural-keyed objects (e.g. audit.AuditLogEntry.actor -> auth.User) - without caching, each one costs a separate DB query. Based on the django forum thread: @@ -135,7 +135,7 @@ class Command(LoadDataCommand): Django's deserializer (deserialize_fk_value) issues a fresh DB query every time it resolves a natural-key FK reference, with no caching of its own. A fixture with many rows referencing the same handful of - natural-keyed objects (e.g. thousands of stock.StockItemTracking rows + natural-keyed objects (e.g. thousands of audit.AuditLogEntry rows all pointing at a few auth.User accounts) would otherwise cost one query per row instead of one query per distinct value. """ diff --git a/src/backend/InvenTree/InvenTree/models.py b/src/backend/InvenTree/InvenTree/models.py index 7f880c053b..017c9a089e 100644 --- a/src/backend/InvenTree/InvenTree/models.py +++ b/src/backend/InvenTree/InvenTree/models.py @@ -547,6 +547,8 @@ class InvenTreeAuditMixin: AUDIT_VIEW_RESTRICTION: Restrict reading audit history to 'staff' or 'superuser' users (or None) AUDIT_REQUIRE_PARENT_PERMISSION: Entries are only visible to users who can view the *parent* model (for models whose access is determined by the linked object, e.g. attachments) + AUDIT_EVENT_TYPES: Enumeration of named event types which can be recorded against this model + (e.g. StockHistoryCode) - see audit.helpers.audit_event """ AUDIT_INCLUDE_FIELDS: tuple[str, ...] = () @@ -556,6 +558,7 @@ class InvenTreeAuditMixin: AUDIT_PARENT_FIELD: Optional[str] = None AUDIT_VIEW_RESTRICTION: Optional[str] = None AUDIT_REQUIRE_PARENT_PERMISSION: bool = False + AUDIT_EVENT_TYPES: Optional[type] = None def get_audit_parent(self): """Return the "parent" object for audit log entries against this instance. diff --git a/src/backend/InvenTree/InvenTree/serializers.py b/src/backend/InvenTree/InvenTree/serializers.py index 4c3d98c4d5..20a94b36c2 100644 --- a/src/backend/InvenTree/InvenTree/serializers.py +++ b/src/backend/InvenTree/InvenTree/serializers.py @@ -1232,7 +1232,7 @@ def apply_duplicate_copy_options( at each call site rather than a new copy-pasted `if duplicate.get(...): instance.copy_..._ from(...)` block. Any duplicate flag whose target method doesn't follow the copy__from() naming convention (e.g. Part's copy_bom/copy_image/copy_tests, or - StockItem's copy_history/copy_tests) must still be handled separately by the caller. + StockItem's copy_tests) must still be handled separately by the caller. Arguments: instance: The newly created instance to copy data onto diff --git a/src/backend/InvenTree/InvenTree/tasks.py b/src/backend/InvenTree/InvenTree/tasks.py index c21b538d67..2110fe9475 100644 --- a/src/backend/InvenTree/InvenTree/tasks.py +++ b/src/backend/InvenTree/InvenTree/tasks.py @@ -346,7 +346,7 @@ def batch_offload_tasks(): Nesting is not supported: a nested batch_offload_tasks() call reuses the outer batch, and only the outermost call schedules a flush. - This mirrors plugin.base.event.events.batch_events() and stock.models.batch_tracking_entries() + This mirrors plugin.base.event.events.batch_events() and audit.helpers.write_batched() - see batch_events()'s docstring for the reasoning behind the on-commit flush and the context-local (rather than parameter-based) design. diff --git a/src/backend/InvenTree/InvenTree/test_commands.py b/src/backend/InvenTree/InvenTree/test_commands.py index 5414c6bde9..139921ce57 100644 --- a/src/backend/InvenTree/InvenTree/test_commands.py +++ b/src/backend/InvenTree/InvenTree/test_commands.py @@ -165,9 +165,8 @@ class CommandTestCase(TestCase): """bulk_create() must not overwrite fixture values for auto_now_add fields. Covers both an auto_now_add DateTimeField (BarcodeScanResult.timestamp, - StockItem.creation_date, StockItemTracking.date) and an auto_now_add - DateField (Part.creation_date) - StockItem/StockItemTracking are also - the models the bug was originally reported against. + StockItem.creation_date) and an auto_now_add DateField (Part.creation_date) + - StockItem is also the model the bug was originally reported against. """ import datetime @@ -176,8 +175,7 @@ class CommandTestCase(TestCase): from common.models import BarcodeScanResult from part.models import Part - from stock.models import StockItem, StockItemTracking - from stock.status_codes import StockHistoryCode + from stock.models import StockItem # JSON fixtures only round-trip datetimes to millisecond precision # (DjangoJSONEncoder truncates microseconds) - use a value already at @@ -207,27 +205,13 @@ class CommandTestCase(TestCase): item.refresh_from_db() self.assertEqual(item.creation_date, original_timestamp) - tracking = StockItemTracking.objects.create( - item=item, tracking_type=StockHistoryCode.CREATED - ) - tracking.date = original_timestamp - tracking.save() - tracking.refresh_from_db() - self.assertEqual(tracking.date, original_timestamp) - - pks = { - 'barcode': entry.pk, - 'part': part.pk, - 'item': item.pk, - 'tracking': tracking.pk, - } + pks = {'barcode': entry.pk, 'part': part.pk, 'item': item.pk} # Serialize parent-before-child, so bulkloaddata's per-model # bulk_create() calls happen in an order that satisfies FK constraints. - data = serializers.serialize('json', [entry, part, item, tracking]) + data = serializers.serialize('json', [entry, part, item]) # Use queryset deletes - Part.delete() refuses to delete an active part - StockItemTracking.objects.filter(pk=tracking.pk).delete() StockItem.objects.filter(pk=item.pk).delete() Part.objects.filter(pk=part.pk).delete() BarcodeScanResult.objects.filter(pk=entry.pk).delete() @@ -247,10 +231,7 @@ class CommandTestCase(TestCase): reloaded_item = StockItem.objects.get(pk=pks['item']) self.assertEqual(reloaded_item.creation_date, original_timestamp) - reloaded_tracking = StockItemTracking.objects.get(pk=pks['tracking']) - self.assertEqual(reloaded_tracking.date, original_timestamp) finally: - StockItemTracking.objects.filter(pk=pks['tracking']).delete() StockItem.objects.filter(pk=pks['item']).delete() Part.objects.filter(pk=pks['part']).delete() BarcodeScanResult.objects.filter(pk=pks['barcode']).delete() diff --git a/src/backend/InvenTree/InvenTree/unit_test.py b/src/backend/InvenTree/InvenTree/unit_test.py index 572f8930cc..6f5044bf7f 100644 --- a/src/backend/InvenTree/InvenTree/unit_test.py +++ b/src/backend/InvenTree/InvenTree/unit_test.py @@ -210,6 +210,48 @@ def findOffloadedEvent( ) +def stock_history(item, **filters): + """Return the audit log entries (stock history) for a StockItem, oldest first. + + Arguments: + item: The StockItem instance (or primary key) - the item may have since been deleted + **filters: Additional filters to apply to the queryset (e.g. event_type=...) + """ + from django.contrib.contenttypes.models import ContentType + + from audit.models import AuditLogEntry + from stock.models import StockItem + + pk = getattr(item, 'pk', item) + + return AuditLogEntry.objects.filter( + content_type=ContentType.objects.get_for_model(StockItem), + object_pk=str(pk), + **filters, + ).order_by('pk') + + +def stock_history_count(**filters) -> int: + """Return the total number of audit log entries recorded for all StockItems. + + Arguments: + **filters: Additional filters to apply (e.g. event_type=...) + """ + from django.contrib.contenttypes.models import ContentType + + from audit.models import AuditLogEntry + from stock.models import StockItem + + return AuditLogEntry.objects.filter( + content_type=ContentType.objects.get_for_model(StockItem), **filters + ).count() + + +def stock_events(item, **filters): + """Return the named stock tracking events recorded for a StockItem, oldest first.""" + return stock_history(item, event_type__isnull=False, **filters) + + class UserMixin: """Mixin to setup a user and login for tests. diff --git a/src/backend/InvenTree/audit/api.py b/src/backend/InvenTree/audit/api.py index c45aef7bdd..ca1a5a7c47 100644 --- a/src/backend/InvenTree/audit/api.py +++ b/src/backend/InvenTree/audit/api.py @@ -22,7 +22,7 @@ from audit.permissions import ( from common.filters import determine_content_type from InvenTree.filters import SEARCH_ORDER_FILTER from InvenTree.helpers_api import InvenTreeApiRouter -from InvenTree.permissions import IsAdminOrAdminScope +from InvenTree.permissions import IsAuthenticatedOrReadScope def audited_model_type_choices(user=None) -> list[tuple[str, str]]: @@ -83,7 +83,7 @@ class AuditLogFilter(rest_filters.FilterSet): """Metaclass options.""" model = AuditLogEntry - fields = ['action', 'actor', 'cid'] + fields = ['action', 'actor', 'cid', 'event_type'] model_type = ModelTypeChoiceFilter( choices=audited_model_type_choices, @@ -178,9 +178,9 @@ class AuditLogViewSet(viewsets.ReadOnlyModelViewSet): ) serializer_class = audit.serializers.AuditLogEntrySerializer - # The audit log is only available to staff users. - # Entries are *additionally* filtered by the user's permissions on each audited model. - permission_classes = [IsAdminOrAdminScope] + # Entries are filtered by the user's permissions on each audited model + # (some models are further restricted to staff users, see audit.registry.AUDIT_VIEW_RESTRICTIONS) + permission_classes = [IsAuthenticatedOrReadScope] filterset_class = AuditLogFilter filter_backends = SEARCH_ORDER_FILTER diff --git a/src/backend/InvenTree/audit/context.py b/src/backend/InvenTree/audit/context.py index 2426d082aa..c988c1bbf8 100644 --- a/src/backend/InvenTree/audit/context.py +++ b/src/backend/InvenTree/audit/context.py @@ -18,6 +18,9 @@ _audit_request = ContextVar('inventree_audit_request', default=None) _audit_cid = ContextVar('inventree_audit_cid', default=None) _audit_source = ContextVar('inventree_audit_source', default=None) _audit_batch = ContextVar('inventree_audit_batch', default=None) +_audit_batch_cid = ContextVar('inventree_audit_batch_cid', default=None) +_audit_events = ContextVar('inventree_audit_events', default=None) +_audit_flush_event = ContextVar('inventree_audit_flush_event', default=None) @contextlib.contextmanager @@ -26,6 +29,9 @@ def audit_batch(): Nested contexts share the outermost list. + The correlation ID is resolved once for the batch (see get_cid), + as resolving it for every entry is expensive for large batches. + Yields: The list of collected entries, which the caller is responsible for writing (or None for a nested context, as there is nothing to write). @@ -36,10 +42,12 @@ def audit_batch(): entries = [] token = _audit_batch.set(entries) + cid_token = _audit_batch_cid.set(resolve_cid()) try: yield entries finally: + _audit_batch_cid.reset(cid_token) _audit_batch.reset(token) @@ -48,6 +56,151 @@ def get_batch() -> Optional[list]: return _audit_batch.get() +class PendingEvent: + """A named event which is waiting to be attached to an audit entry (see audit.helpers.audit_event).""" + + def __init__(self, instance, event_type: int, notes=None, deltas=None, actor=None): + """Initialize the pending event.""" + self.instance = instance + self.event_type = event_type + self.notes = notes + self.deltas = deltas + self.actor = actor + self.consumed = False + + +def instance_key(instance) -> Optional[tuple]: + """Return a key which identifies the database row of an instance (or None if unsaved).""" + if instance.pk is None: + return None + + return (type(instance), instance.pk) + + +class EventRegistry: + """The pending events within an audit_event() context. + + Events are indexed by instance (identity), and by database row (model and primary key), + so that matching events can be found efficiently for large numbers of events. + """ + + def __init__(self): + """Initialize an empty registry.""" + self.counter = 0 + self.by_id: dict[int, list] = {} + self.by_key: dict[tuple, list] = {} + + def add(self, event: PendingEvent) -> None: + """Add an event to the registry.""" + self.counter += 1 + event.order = self.counter + + self.by_id.setdefault(id(event.instance), []).append(event) + + if key := instance_key(event.instance): + self.by_key.setdefault(key, []).append(event) + + def remove(self, event: PendingEvent) -> None: + """Remove an event from the registry.""" + for index, key in [ + (self.by_id, id(event.instance)), + (self.by_key, instance_key(event.instance)), + ]: + if key is not None and event in (events := index.get(key, [])): + events.remove(event) + + if not events: + del index[key] + + def find(self, instance) -> Optional[PendingEvent]: + """Return the first (unconsumed) pending event for the provided instance. + + Instances are matched by identity, or by model and primary key + (e.g. if the same database row is saved via a different instance). + """ + candidates = list(self.by_id.get(id(instance), [])) + + if key := instance_key(instance): + candidates += self.by_key.get(key, []) + + candidates = [event for event in candidates if not event.consumed] + + return min(candidates, key=lambda event: event.order, default=None) + + +@contextlib.contextmanager +def pending_events(events: list): + """Register the provided events as pending for the duration of the context.""" + registry = _audit_events.get() + token = None + + if registry is None: + registry = EventRegistry() + token = _audit_events.set(registry) + + for event in events: + registry.add(event) + + try: + yield + finally: + for event in events: + registry.remove(event) + + if token is not None: + _audit_events.reset(token) + + +def has_pending_event(instance) -> bool: + """Return True if there is a pending (unconsumed) event for the provided instance.""" + registry = _audit_events.get() + return registry is not None and registry.find(instance) is not None + + +@contextlib.contextmanager +def flushing_event(event: PendingEvent): + """Consume the provided event for any entry recorded for its instance within this context. + + Used when writing an entry for a specific event (see audit.helpers.write_event_entry), + which may not be the first pending event for the instance. + """ + token = _audit_flush_event.set(event) + + try: + yield + finally: + _audit_flush_event.reset(token) + + +def consume_event(instance) -> Optional[PendingEvent]: + """Return (and consume) the first pending event for the provided instance, if any.""" + event = _audit_flush_event.get() + + if ( + event is not None + and not event.consumed + and ( + event.instance is instance + or ( + (key := instance_key(instance)) is not None + and key == instance_key(event.instance) + ) + ) + ): + event.consumed = True + return event + + registry = _audit_events.get() + + if registry is None: + return None + + if event := registry.find(instance): + event.consumed = True + + return event + + @contextlib.contextmanager def audit_request(request): """Track the provided request (and a new correlation ID) for the duration of the context.""" @@ -131,6 +284,17 @@ def end_worker_task() -> None: def get_cid() -> Optional[str]: """Return the current correlation ID (used as AUDITLOG_CID_GETTER). + Within an audit_batch() context, the ID resolved when the batch was entered is returned. + """ + if cid := _audit_batch_cid.get(): + return cid + + return resolve_cid() + + +def resolve_cid() -> Optional[str]: + """Resolve the current correlation ID. + - Within a request: the request ID bound by django-structlog, so that audit entries can be matched against log entries (and the X-Request-ID header) for the request. Note: django-structlog uses the incoming 'X-Request-ID' header if provided diff --git a/src/backend/InvenTree/audit/helpers.py b/src/backend/InvenTree/audit/helpers.py index dfcdfe6b03..3c37539bbd 100644 --- a/src/backend/InvenTree/audit/helpers.py +++ b/src/backend/InvenTree/audit/helpers.py @@ -1,11 +1,15 @@ -"""Helper functions for recording audit entries for bulk database operations. +"""Helper functions for recording audit entries. -QuerySet.update() (and bulk_create / bulk_update) bypass model signals, -so changes made this way are not recorded by django-auditlog. -Code which modifies audited models in bulk should use these helpers instead. +- Named events (with notes and deltas) can be recorded against an instance (audit_event / log_event) +- QuerySet.update() (and bulk_create / bulk_update) bypass model signals, + so changes made this way are not recorded by django-auditlog. + Code which modifies audited models in bulk should use the bulk helpers instead. """ import contextlib +import datetime +from decimal import Decimal +from typing import Optional from django.conf import settings from django.core.exceptions import FieldDoesNotExist @@ -14,6 +18,7 @@ from django.db.models.signals import pre_save from auditlog.context import auditlog_disabled from auditlog.registry import auditlog +from auditlog.signals import pre_log def is_audited(model) -> bool: @@ -108,7 +113,13 @@ def audited_update(queryset, **values) -> int: with transaction.atomic(): if connection.features.has_select_for_update: - old_rows = list(queryset.select_for_update()) + # Only lock rows of the model itself (related rows may be joined via select_related) + if connection.features.has_select_for_update_of: + locked = queryset.select_for_update(of=('self',)) + else: + locked = queryset.select_for_update() + + old_rows = list(locked) else: old_rows = list(queryset) @@ -125,6 +136,9 @@ def audited_update(queryset, **values) -> int: with write_batched(): for old in old_rows: if new := new_rows.get(old.pk): + # Retain cached related objects (e.g. from select_related) + copy_related_cache(old, new) + _create_log_entry( action=AuditLogEntry.Action.UPDATE, instance=new, @@ -210,9 +224,6 @@ def audited_bulk_create(model, objs: list) -> list: Returns: list: The created objects, as fetched from the database (in insertion order) """ - from auditlog.receivers import _create_log_entry - - from audit.models import AuditLogEntry from InvenTree.helpers_db import bulk_create_and_fetch # Required by bulk_create_and_fetch() - checked for all models (not only audited ones), @@ -241,16 +252,176 @@ def audited_bulk_create(model, objs: list) -> list: obj._state.adding = False obj._state.db = instance._state.db - if is_audited(model): - with write_batched(): - for instance in created: - _create_log_entry( - action=AuditLogEntry.Action.CREATE, - instance=instance, - sender=model, - diff_old=None, - diff_new=instance, - use_json_for_changes=settings.AUDITLOG_STORE_JSON_CHANGES, - ) + record_created(model, created) return created + + +def record_created(model, instances) -> None: + """Record a CREATE audit entry for each of the provided (newly created) instances. + + For instances which have been created in bulk (bypassing model signals), + e.g. via InvenTree.helpers_db.bulk_create_and_fetch(). + The entries are written with a single bulk insert. + + Note: Any pending events for the instances (see audit_event) are attached to the entries. + """ + from auditlog.receivers import _create_log_entry + + from audit.models import AuditLogEntry + + if not is_audited(model): + return + + with transaction.atomic(), write_batched(): + for instance in instances: + _create_log_entry( + action=AuditLogEntry.Action.CREATE, + instance=instance, + sender=model, + diff_old=None, + diff_new=instance, + use_json_for_changes=settings.AUDITLOG_STORE_JSON_CHANGES, + ) + + +def audit_reference(obj) -> dict: + """Return a reference to a model instance, for recording in event deltas. + + The string representation is recorded at the time of the event, + so that the reference remains meaningful if the object is later changed or deleted. + """ + return { + 'model_type': f'{obj._meta.app_label}.{obj._meta.model_name}', + 'pk': obj.pk, + 'str': str(obj), + } + + +def serialize_deltas(value): + """Convert event deltas into a JSON serializable form. + + - Model instances are converted to references (see audit_reference) + - Decimal values are converted to float + - Dates are converted to ISO format strings + """ + from django.db import models + + if isinstance(value, models.Model): + return audit_reference(value) + + if isinstance(value, dict): + return {str(key): serialize_deltas(val) for key, val in value.items()} + + if isinstance(value, (list, tuple, set)): + return [serialize_deltas(val) for val in value] + + if isinstance(value, Decimal): + return float(value) + + if isinstance(value, (datetime.date, datetime.datetime)): + return value.isoformat() + + return value + + +def validate_event_type(model, event_type: int) -> int: + """Ensure that the event type is valid for the given model.""" + from audit.registry import event_type_choices + + if event_type is None: + raise ValueError('An event type must be provided') + + if int(event_type) not in event_type_choices(model): + raise ValueError( + f"Invalid audit event type '{event_type}' for model {model._meta.label}" + ) + + return int(event_type) + + +@contextlib.contextmanager +def audit_event( + instance, event_type: Optional[int], notes: str = '', deltas=None, actor=None +): + """Record a named event against an instance. + + The first audit entry recorded for the instance within this context + (e.g. when the instance is saved) carries the event type, notes and deltas. + + If no entry is recorded for the instance (e.g. no fields were changed, or the + instance was not saved), an entry is created for the event when the context exits. + + If no event type is provided, the notes / deltas / actor are attached to any entry + recorded for the instance within this context - but no entry is created otherwise. + + Contexts can be nested, to record events against multiple instances. + + Arguments: + instance: The (audited) model instance + event_type: The event type (must be defined in the AUDIT_EVENT_TYPES of the model), or None + notes: Optional notes associated with the event + deltas: Optional dict of additional information (e.g. references to related objects) + Deltas which are also recorded as field changes (of the same entry) are omitted. + actor: Optional user who performed the event (defaults to the user of the current request) + + Yields: + The pending event (consumed once it has been attached to an audit entry) + """ + from audit.context import PendingEvent, pending_events + from audit.models import NOTES_MAX_LENGTH + + if event_type is not None: + event_type = validate_event_type(type(instance), event_type) + + event = PendingEvent( + instance, + event_type, + notes=str(notes)[:NOTES_MAX_LENGTH] if notes else None, + deltas=serialize_deltas(deltas) if deltas else None, + actor=actor, + ) + + with pending_events([event]): + yield event + + # An entry is only created without any changes for a *named* event + if not event.consumed and event.event_type is not None: + write_event_entry(event) + + +def write_event_entry(event) -> None: + """Write an audit entry for an event which has not been attached to any other entry.""" + from audit.context import flushing_event + from audit.models import AuditLogEntry + + instance = event.instance + + # The instance was never saved (or has since been deleted) + if instance.pk is None or not is_audited(type(instance)): + return + + action = AuditLogEntry.Action.UPDATE + + # Apply the same checks as for any other entry (e.g. audit log disabled) + results = pre_log.send(type(instance), instance=instance, action=action) + + if any(result is False for _, result in results): + return + + # Attach this event to the entry - not the first pending event for the instance, + # which may belong to an (outer) context which has not yet exited + with flushing_event(event): + AuditLogEntry.objects.log_create( + instance, action=action, changes={}, force_log=True + ) + + +def log_event( + instance, event_type: int, notes: str = '', deltas=None, actor=None +) -> None: + """Record a named event against an instance, which does not involve saving the instance.""" + validate_event_type(type(instance), event_type) + + with audit_event(instance, event_type, notes=notes, deltas=deltas, actor=actor): + pass diff --git a/src/backend/InvenTree/audit/migrations/0003_event_type.py b/src/backend/InvenTree/audit/migrations/0003_event_type.py new file mode 100644 index 0000000000..ca07e50ebb --- /dev/null +++ b/src/backend/InvenTree/audit/migrations/0003_event_type.py @@ -0,0 +1,31 @@ +# Generated by Django 5.2.17 on 2026-10-25 02:32 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ("audit", "0002_actor_username"), + ] + + operations = [ + migrations.AddField( + model_name="auditlogentry", + name="event_type", + field=models.PositiveIntegerField( + blank=True, null=True, verbose_name="Event Type" + ), + ), + migrations.AddField( + model_name="auditlogentry", + name="notes", + field=models.CharField( + blank=True, + help_text="Notes associated with this entry", + max_length=512, + null=True, + verbose_name="Notes", + ), + ), + ] diff --git a/src/backend/InvenTree/audit/models.py b/src/backend/InvenTree/audit/models.py index f17501873e..1e11d481b8 100644 --- a/src/backend/InvenTree/audit/models.py +++ b/src/backend/InvenTree/audit/models.py @@ -1,6 +1,7 @@ """Database models for the audit app.""" import os +from typing import Optional from django.contrib.contenttypes.models import ContentType from django.core.exceptions import FieldDoesNotExist @@ -14,6 +15,9 @@ import audit.context MASKED_VALUE = '***' +# Maximum length of the notes associated with an audit entry +NOTES_MAX_LENGTH = 512 + def get_audit_parent_reference(instance): """Return a (ContentType, pk) reference to the audit "parent" of an instance (or None).""" @@ -161,7 +165,13 @@ class AuditLogEntryManager(LogEntryManager): kwargs.setdefault('parent_content_type', parent_ct) kwargs.setdefault('parent_object_pk', parent_pk) - if 'actor' not in kwargs and (actor := audit.context.get_request_actor()): + # A pending named event for this instance (see audit.helpers.audit_event) + event = audit.context.consume_event(instance) + + # The actor is (in order of preference) provided explicitly, by the event, or by the request + if 'actor' not in kwargs and ( + actor := (event and event.actor) or audit.context.get_request_actor() + ): kwargs['actor'] = actor kwargs['actor_email'] = getattr(actor, 'email', None) @@ -179,12 +189,35 @@ class AuditLogEntryManager(LogEntryManager): kwargs['additional_data'] = {**(additional_data or {}), 'source': source} + # Attach the pending named event + if event: + kwargs['event_type'] = event.event_type + kwargs['notes'] = event.notes or None + + # Deltas which are already recorded as field changes are not duplicated + changes = kwargs.get('changes') + deltas = { + key: value + for key, value in (event.deltas or {}).items() + if not (isinstance(changes, dict) and key in changes) + } + + if deltas: + kwargs['additional_data'] = { + **(kwargs.get('additional_data') or {}), + 'deltas': deltas, + } + is_masked = getattr(instance, 'is_audit_masked', None) if callable(is_masked) and is_masked(): # Mask at write time - sensitive values must never reach the log table kwargs['changes'] = mask_changes(kwargs.get('changes')) kwargs['serialized_data'] = None + kwargs['notes'] = None + + if additional_data := kwargs.get('additional_data'): + additional_data.pop('deltas', None) return kwargs @@ -324,6 +357,29 @@ class AuditLogEntry(AbstractLogEntry): max_length=150, null=True, blank=True, verbose_name=_('Actor Username') ) + # Named event type (see InvenTreeAuditMixin.AUDIT_EVENT_TYPES) + event_type = models.PositiveIntegerField( + null=True, blank=True, verbose_name=_('Event Type') + ) + + notes = models.CharField( + max_length=NOTES_MAX_LENGTH, + null=True, + blank=True, + verbose_name=_('Notes'), + help_text=_('Notes associated with this entry'), + ) + + @property + def event_label(self) -> Optional[str]: + """Return the label for the event type of this entry (if any).""" + from audit.registry import event_type_label + + if self.event_type is None: + return None + + return event_type_label(self.content_type.model_class(), self.event_type) + @property def model_type(self) -> str: """Return the 'app_label.model' string for the audited model.""" diff --git a/src/backend/InvenTree/audit/registry.py b/src/backend/InvenTree/audit/registry.py index 49c1ee114f..92ffae4d5e 100644 --- a/src/backend/InvenTree/audit/registry.py +++ b/src/backend/InvenTree/audit/registry.py @@ -1,5 +1,8 @@ """Registration of audited models with django-auditlog.""" +import functools +from typing import Optional + from django.apps import apps import structlog @@ -103,3 +106,42 @@ def get_view_restriction(model): return restriction return AUDIT_VIEW_RESTRICTIONS.get(model._meta.label_lower) + + +def event_type_choices(model) -> dict[int, str]: + """Return a mapping of event type value -> label for the given model. + + Event types are defined by the AUDIT_EVENT_TYPES attribute of the model, + which may be a StatusCode enumeration or a Django Choices class. + """ + enum = getattr(model, 'AUDIT_EVENT_TYPES', None) + + if enum is None: + return {} + + return _enum_choices(enum) + + +@functools.cache +def _enum_choices(enum) -> dict[int, str]: + """Return a mapping of value -> label for an event type enumeration (cached). + + Note: Labels are returned as (lazy) translatable strings, + so that they are translated at the point of use. + """ + # StatusCode enumeration (e.g. StockHistoryCode) + if callable(getattr(enum, 'items', None)): + return dict(enum.items(custom=False)) + + # Django Choices class + return {int(key): label for key, label in enum.choices} + + +def event_type_label(model, event_type: int) -> Optional[str]: + """Return the label for an event type of the given model (or None if unknown).""" + if model is None or event_type is None: + return None + + label = event_type_choices(model).get(int(event_type)) + + return None if label is None else str(label) diff --git a/src/backend/InvenTree/audit/serializers.py b/src/backend/InvenTree/audit/serializers.py index 9ebbf1ea08..6e41f62c2f 100644 --- a/src/backend/InvenTree/audit/serializers.py +++ b/src/backend/InvenTree/audit/serializers.py @@ -81,6 +81,10 @@ class AuditLogEntrySerializer( 'actor', 'actor_detail', 'actor_username', + 'event_type', + 'event_label', + 'notes', + 'deltas', 'model_type', 'model_label', 'object_pk', @@ -115,6 +119,42 @@ class AuditLogEntrySerializer( read_only=True, allow_null=True, label=_('Parent Model Type') ) + event_label = serializers.CharField( + read_only=True, allow_null=True, label=_('Event') + ) + + deltas = serializers.SerializerMethodField(label=_('Deltas')) + + @extend_schema_field(serializers.JSONField(allow_null=True)) + def get_deltas(self, entry: AuditLogEntry) -> Optional[dict]: + """Return the deltas recorded against a named event. + + References to objects which the user cannot view are redacted. + """ + deltas = (entry.additional_data or {}).get('deltas') + + if not isinstance(deltas, dict): + return None + + return {key: self.redact_reference(value) for key, value in deltas.items()} + + def redact_reference(self, value): + """Redact a reference to a model instance, if the user cannot view the model.""" + from django.apps import apps + + if not (isinstance(value, dict) and 'model_type' in value and 'pk' in value): + return value + + try: + model = apps.get_model(value['model_type']) + except (LookupError, ValueError): + model = None + + if self.can_view_model(model): + return value + + return {**value, 'pk': None, 'str': None} + changes = serializers.SerializerMethodField(label=_('Changes')) changes_detail = OptionalField( @@ -297,6 +337,15 @@ class AuditLogEntrySerializer( for field in self.STAFF_ONLY_FIELDS: data.pop(field, None) + # Event deltas are provided (with redaction) via the 'deltas' field + if isinstance(data.get('additional_data'), dict): + additional_data = { + key: value + for key, value in data['additional_data'].items() + if key != 'deltas' + } + data['additional_data'] = additional_data or None + # Only expose the parent reference if the user may view the parent model if parent_ct := instance.parent_content_type: if not self.can_view_model(parent_ct.model_class()): diff --git a/src/backend/InvenTree/audit/tests.py b/src/backend/InvenTree/audit/tests.py index 631b3defa7..86c96e55f5 100644 --- a/src/backend/InvenTree/audit/tests.py +++ b/src/backend/InvenTree/audit/tests.py @@ -7,10 +7,12 @@ PurchaseOrder (parent) and PurchaseOrderLineItem (child) with the audit log. import contextlib from base64 import b64encode from datetime import timedelta +from decimal import Decimal from unittest import mock from django.contrib.auth import get_user_model from django.contrib.contenttypes.models import ContentType +from django.db import models, transaction from django.test import TestCase, override_settings from django.urls import reverse from django.utils import timezone @@ -18,6 +20,7 @@ from django.utils import timezone from auditlog.registry import auditlog import audit.context +from audit.helpers import write_batched from audit.models import MASKED_VALUE, AuditLogEntry, mask_changes from audit.registry import BASE_EXCLUDE_FIELDS, audited_models, register_model from audit.serializers import AuditLogEntrySerializer @@ -145,6 +148,7 @@ class AuditRegistryTest(TestCase): 'pricing.partinternalpricebreak', 'pricing.partpricing', 'pricing.partsellpricebreak', + 'stock.stockitem', ], ) @@ -269,6 +273,32 @@ class AuditModelTest(AuditTestMixin, TestCase): self.assertIsNone(audit.context.get_source()) self.assertIsNone(audit.context.get_cid()) + def test_batch_cid(self): + """The correlation ID is resolved once for an audit batch.""" + with audit.context.audit_source('task:test_task'): + cid = audit.context.get_cid() + + with ( + mock.patch.object( + audit.context, 'resolve_cid', wraps=audit.context.resolve_cid + ) as resolve, + transaction.atomic(), + write_batched(), + ): + for idx in range(3): + self.po.description = f'Batched {idx}' + self.po.save() + + self.assertEqual(resolve.call_count, 1) + + entries = self.entries_for(self.po, action=AuditLogEntry.Action.UPDATE) + + self.assertEqual(entries.count(), 3) + self.assertEqual({entry.cid for entry in entries}, {cid}) + + # Context is reset afterwards + self.assertIsNone(audit.context.get_cid()) + def test_disabled(self): """No entries are recorded when the audit log is disabled.""" set_global_setting('AUDIT_LOG_ENABLED', False) @@ -536,8 +566,8 @@ class AuditAPITest(AuditTestMixin, InvenTreeAPITestCase): len(line_entries({'model_type': 'purchaseorderlineitem'})), 1 ) - def test_staff_required(self): - """The audit log API is only available to staff users.""" + def test_non_staff(self): + """Non-staff users can view entries for models they have permission to view.""" self.patch_po('Edit') entry = self.entries_for(self.po).first() @@ -548,12 +578,24 @@ class AuditAPITest(AuditTestMixin, InvenTreeAPITestCase): self.assertEqual(data[0]['remote_addr'], '127.0.0.1') self.get(detail_url) - # Non-staff users are denied - even with view permission on the audited model self.user.is_staff = False self.user.save() - self.get(self.list_url(), expected_code=403) - self.get(detail_url, expected_code=403) + # Non-staff users can view the entries - but not the remote address + data = self.po_history() + self.assertGreater(len(data), 0) + self.assertNotIn('remote_addr', data[0]) + self.assertNotIn('remote_addr', self.get(detail_url).data) + + # ...but not without view permission on the audited model + self.clearRoles() + + self.assertEqual(self.po_history(), []) + self.get(detail_url, expected_code=404) + + # Anonymous users are denied + self.client.logout() + self.get(self.list_url(), expected_code=401) def test_view_restriction(self): """Models with a view restriction are hidden from users without that status.""" @@ -1259,3 +1301,305 @@ class AuditRetentionTest(InvenTreeAPITestCase): data = self.get(url).data self.assertIsNone(data['actor']) self.assertEqual(data['actor_username'], 'leaver') + + +class AuditTestEvents(models.IntegerChoices): + """Event types used for testing named audit events.""" + + MOVED = 10, 'Moved' + COUNTED = 20, 'Counted' + + +class AuditEventTest(InvenTreeAPITestCase): + """Tests for named audit events (see audit.helpers.audit_event).""" + + roles = ['purchase_order.view', 'purchase_order.change'] + + @classmethod + def setUpClass(cls): + """Define event types for the Company model.""" + cls.events_patch = mock.patch.object( + Company, 'AUDIT_EVENT_TYPES', AuditTestEvents + ) + cls.events_patch.start() + + super().setUpClass() + + @classmethod + def tearDownClass(cls): + """Remove event types from the Company model.""" + super().tearDownClass() + cls.events_patch.stop() + + @classmethod + def setUpTestData(cls): + """Create test data.""" + super().setUpTestData() + + cls.company = Company.objects.create(name='Event Company', is_supplier=True) + cls.part = Part.objects.create(name='Event Part', description='A part') + + def entries(self): + """Return all entries for the test company (oldest first).""" + return AuditLogEntry.objects.get_for_object(self.company).order_by('pk') + + def test_event_with_changes(self): + """The event is attached to the entry recorded when the instance is saved.""" + from audit.helpers import audit_event + + n = self.entries().count() + + with audit_event( + self.company, + AuditTestEvents.MOVED, + notes='Moved company', + deltas={'part': self.part, 'quantity': Decimal('1.5')}, + ): + self.company.description = 'Moved' + self.company.save() + + # A single entry, which records both the changes and the event + self.assertEqual(self.entries().count(), n + 1) + + entry = self.entries().last() + + self.assertEqual(entry.event_type, AuditTestEvents.MOVED) + self.assertEqual(entry.event_label, 'Moved') + self.assertEqual(entry.notes, 'Moved company') + self.assertEqual(entry.changes_dict['description'], ['', 'Moved']) + self.assertEqual( + entry.additional_data['deltas'], + { + 'part': { + 'model_type': 'part.part', + 'pk': self.part.pk, + 'str': str(self.part), + }, + 'quantity': 1.5, + }, + ) + + def test_event_without_changes(self): + """An event is recorded even if no fields are changed.""" + from audit.helpers import audit_event, log_event + + n = self.entries().count() + + # Saved, but not changed + with audit_event(self.company, AuditTestEvents.COUNTED, notes='Counted'): + self.company.save() + + # Not saved at all + log_event(self.company, AuditTestEvents.COUNTED, deltas={'quantity': 5}) + + entries = list(self.entries()[n:]) + + self.assertEqual(len(entries), 2) + + for entry in entries: + self.assertEqual(entry.action, AuditLogEntry.Action.UPDATE) + self.assertEqual(entry.event_type, AuditTestEvents.COUNTED) + self.assertEqual(entry.changes_dict, {}) + + self.assertEqual(entries[0].notes, 'Counted') + self.assertEqual(entries[1].additional_data['deltas'], {'quantity': 5}) + + def test_unnamed_event(self): + """Notes (without an event type) are attached to an entry - but do not create one.""" + from audit.helpers import audit_event + + n = self.entries().count() + + # No changes - no entry + with audit_event(self.company, None, notes='Nothing changed'): + self.company.save() + + self.assertEqual(self.entries().count(), n) + + with audit_event(self.company, None, notes='Changed'): + self.company.description = 'Unnamed' + self.company.save() + + [entry] = list(self.entries()[n:]) + + self.assertIsNone(entry.event_type) + self.assertIsNone(entry.event_label) + self.assertEqual(entry.notes, 'Changed') + + def test_no_changes_without_event(self): + """Saving an instance without changes (and without an event) is not recorded.""" + n = self.entries().count() + self.company.save() + self.assertEqual(self.entries().count(), n) + + def test_multiple_instances(self): + """Events can be recorded against multiple instances (one entry each).""" + from audit.helpers import audit_event + + other = Company.objects.create(name='Other Company') + + with ( + audit_event(self.company, AuditTestEvents.MOVED, deltas={'to': other}), + audit_event(other, AuditTestEvents.COUNTED, deltas={'from': self.company}), + ): + self.company.description = 'A' + self.company.save() + + entry = self.entries().last() + self.assertEqual(entry.event_type, AuditTestEvents.MOVED) + self.assertEqual(entry.additional_data['deltas']['to']['pk'], other.pk) + + # The other instance was not saved - so an entry is created for the event + other_entry = AuditLogEntry.objects.get_for_object(other).order_by('pk').last() + self.assertEqual(other_entry.event_type, AuditTestEvents.COUNTED) + self.assertEqual(other_entry.changes_dict, {}) + + def test_nested_events_same_instance(self): + """Nested events for the same (unsaved) instance each create an entry.""" + from contextlib import ExitStack + + from audit.helpers import audit_event + + n = self.entries().count() + + # Contexts exit in reverse order (newest event first) + with ExitStack() as stack: + for quantity in [1, 2]: + stack.enter_context( + audit_event( + self.company, + AuditTestEvents.COUNTED, + deltas={'quantity': quantity}, + ) + ) + + entries = list(self.entries()[n:]) + + self.assertEqual(len(entries), 2) + self.assertEqual( + sorted(entry.additional_data['deltas']['quantity'] for entry in entries), + [1, 2], + ) + + def test_new_instance(self): + """An event can be attached to the creation of a new instance.""" + from audit.helpers import audit_event + + company = Company(name='New Company') + + with audit_event(company, AuditTestEvents.MOVED, notes='Created'): + company.save() + + [entry] = AuditLogEntry.objects.get_for_object(company) + + self.assertEqual(entry.action, AuditLogEntry.Action.CREATE) + self.assertEqual(entry.event_type, AuditTestEvents.MOVED) + self.assertEqual(entry.notes, 'Created') + + def test_invalid_event_type(self): + """Event types must be defined by the model.""" + from audit.helpers import log_event + + with self.assertRaises(ValueError): + log_event(self.company, 999) + + with self.assertRaises(ValueError): + log_event(self.company, None) + + # Models without event types + with self.assertRaises(ValueError): + log_event(self.part, AuditTestEvents.MOVED) + + def test_error_not_recorded(self): + """No event entry is created if an error occurs within the context.""" + from audit.helpers import audit_event + + n = self.entries().count() + + with ( + self.assertRaises(RuntimeError), + audit_event(self.company, AuditTestEvents.MOVED), + ): + raise RuntimeError('Failed') + + self.assertEqual(self.entries().count(), n) + + def test_disabled(self): + """Events are not recorded when auditing is disabled.""" + from auditlog.context import disable_auditlog + + from audit.helpers import log_event + + n = self.entries().count() + + with disable_auditlog(): + log_event(self.company, AuditTestEvents.MOVED) + + set_global_setting('AUDIT_LOG_ENABLED', False) + self.addCleanup(set_global_setting, 'AUDIT_LOG_ENABLED', True) + + log_event(self.company, AuditTestEvents.MOVED) + + self.assertEqual(self.entries().count(), n) + + def test_batched(self): + """Event entries can be written in a batch.""" + from django.db import transaction + + from audit.helpers import log_event, write_batched + + n = self.entries().count() + + with transaction.atomic(), write_batched(): + log_event(self.company, AuditTestEvents.MOVED) + log_event(self.company, AuditTestEvents.COUNTED) + + # Not written until the batch is complete + self.assertEqual(self.entries().count(), n) + + self.assertEqual( + [entry.event_type for entry in self.entries()[n:]], + [AuditTestEvents.MOVED, AuditTestEvents.COUNTED], + ) + + def test_api(self): + """Event information is provided via the API (with redacted references).""" + from audit.helpers import log_event + + log_event( + self.company, + AuditTestEvents.MOVED, + notes='Via API', + deltas={'part': self.part, 'quantity': 3}, + ) + + entry = self.entries().last() + url = reverse('api-audit-log-list') + + # Filter by event type + data = self.get(url, {'event_type': AuditTestEvents.MOVED}).data + self.assertEqual([item['pk'] for item in data], [entry.pk]) + self.assertEqual( + self.get(url, {'event_type': AuditTestEvents.COUNTED}).data, [] + ) + + [item] = data + + self.assertEqual(item['event_type'], AuditTestEvents.MOVED) + self.assertEqual(item['event_label'], 'Moved') + self.assertEqual(item['notes'], 'Via API') + self.assertEqual(item['deltas']['quantity'], 3) + + # Deltas are only provided via the 'deltas' field + self.assertIsNone(item['additional_data']) + + # The user cannot view parts - the reference is redacted + self.assertEqual( + item['deltas']['part'], {'model_type': 'part.part', 'pk': None, 'str': None} + ) + + self.assignRole('part.view') + + item = self.get(url, {'event_type': AuditTestEvents.MOVED}).data[0] + self.assertEqual(item['deltas']['part']['pk'], self.part.pk) + self.assertEqual(item['deltas']['part']['str'], str(self.part)) diff --git a/src/backend/InvenTree/build/models.py b/src/backend/InvenTree/build/models.py index 02ee6b91f5..b23d9a35f0 100644 --- a/src/backend/InvenTree/build/models.py +++ b/src/backend/InvenTree/build/models.py @@ -2,6 +2,7 @@ import copy import decimal +from contextlib import ExitStack from typing import Optional, TypedDict from django.contrib.auth.models import User @@ -31,6 +32,13 @@ import part.models import report.mixins import stock.models import users.models +from audit.helpers import ( + audit_event, + audited_bulk_update, + copy_related_cache, + record_created, + write_batched, +) from build.events import BuildEvents from build.filters import annotate_allocated_quantity, annotate_required_quantity from build.status_codes import BuildStatus, BuildStatusGroups @@ -687,7 +695,7 @@ class Build( Notes: This unrolls what would otherwise be a per-BuildItem call, so that the underlying StockItem, - StockItemTracking, BuildLine and BuildItem writes can be batched into + audit log, BuildLine and BuildItem writes can be batched into a handful of bulk queries instead of several per build item. """ import part.tasks @@ -704,7 +712,15 @@ class Build( # Preselect related fields to avoid per-row database queries below build_items = list( build_items.select_related( - 'build_line', 'stock_item', 'stock_item__part', 'install_into' + 'build_line', + 'stock_item', + 'stock_item__part', + 'stock_item__location', + 'stock_item__purchase_order__supplier', + 'install_into', + 'install_into__part', + 'install_into__location', + 'install_into__purchase_order__supplier', ) ) @@ -843,8 +859,10 @@ class Build( # which the tracking entries and (for installed items) BuildItem records need below new_stock_item_data = [new_item for _, new_item, _ in split_items] - new_stock_items = bulk_create_and_fetch( - stock.models.StockItem, new_stock_item_data + # Evaluate the queryset immediately (into a list) - otherwise, indexing into it + # below would re-query the database once per split item + new_stock_items = list( + bulk_create_and_fetch(stock.models.StockItem, new_stock_item_data) ) # Back-populate the resolved primary keys onto the original 'new_item' objects @@ -853,68 +871,23 @@ class Build( # resolved pk too. On backends that can't return pks from bulk_create (MySQL), # 'new_stock_items' are freshly-fetched, distinct objects, so a plain list # replacement here would leave those other references with pk=None. - for i, (_stock_item, new_item, _quantity) in enumerate(split_items): + for i, (source_item, new_item, _quantity) in enumerate(split_items): new_item.pk = new_stock_items[i].pk + new_item._state.adding = False + + # Retain the cached related objects (avoid per-item lookups) + copy_related_cache(source_item, new_item) + copy_related_cache(source_item, new_stock_items[i]) - tracking_entries = [] split_events = [] install_events = [] # Split stock items for "split_items" - for source_item, new_item, quantity in split_items: - tracking_entries.append( - stock.models.StockItemTracking( - item_id=new_item.pk, - part_id=new_item.part_id, - tracking_type=StockHistoryCode.SPLIT_FROM_PARENT.value, - user=user, - notes=notes, - deltas={'stockitem': source_item.pk, 'quantity': float(quantity)}, - ) - ) - tracking_entries.append( - stock.models.StockItemTracking( - item_id=source_item.pk, - part_id=source_item.part_id, - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value, - user=user, - notes=notes, - deltas={ - 'removed': float(quantity), - 'quantity': float(source_item.quantity), - }, - ) - ) - + for source_item, new_item, _quantity in split_items: split_events.append(((), {'id': new_item.pk, 'parent': source_item.pk})) # Install stock items for "install_items" - for build_item, target_item, output, quantity in install_items: - tracking_entries.append( - stock.models.StockItemTracking( - item_id=target_item.pk, - part_id=target_item.part_id, - tracking_type=StockHistoryCode.INSTALLED_INTO_ASSEMBLY.value, - user=user, - notes=notes, - deltas={ - 'stockitem': output.pk, - 'quantity': float(quantity), - 'buildorder': self.pk, - }, - ) - ) - tracking_entries.append( - stock.models.StockItemTracking( - item_id=output.pk, - part_id=output.part_id, - tracking_type=StockHistoryCode.INSTALLED_CHILD_ITEM.value, - user=user, - notes=notes, - deltas={'stockitem': target_item.pk, 'quantity': float(quantity)}, - ) - ) - + for build_item, target_item, output, _quantity in install_items: install_events.append(( (), {'id': target_item.pk, 'assembly_id': output.pk}, @@ -923,27 +896,55 @@ class Build( # Ensure the build item points to the (possibly newly split) stock item build_item.stock_item = target_item - # Consume stock items for "consume_items" - for target_item, quantity in consume_items: - tracking_entries.append( - stock.models.StockItemTracking( - item_id=target_item.pk, - part_id=target_item.part_id, - tracking_type=StockHistoryCode.BUILD_CONSUMED.value, - user=user, - notes=notes, - deltas={'buildorder': self.pk, 'quantity': float(quantity)}, + # Record the operation in the audit log - a single entry for each stock item + with write_batched(), ExitStack() as events: + + def audit(item, code, deltas): + """Record a named event against the provided stock item.""" + events.enter_context( + audit_event(item, code, notes=notes, deltas=deltas, actor=user) ) + + for source_item, new_item, quantity in split_items: + audit( + source_item, + StockHistoryCode.SPLIT_CHILD_ITEM, + { + 'stockitem': new_item, + 'removed': quantity, + 'quantity': source_item.quantity, + }, + ) + + for _build_item, target_item, output, quantity in install_items: + audit( + target_item, + StockHistoryCode.INSTALLED_INTO_ASSEMBLY, + {'stockitem': output, 'quantity': quantity, 'buildorder': self}, + ) + audit( + output, + StockHistoryCode.INSTALLED_CHILD_ITEM, + {'stockitem': target_item, 'quantity': quantity}, + ) + + for target_item, quantity in consume_items: + audit( + target_item, + StockHistoryCode.BUILD_CONSUMED, + {'buildorder': self, 'quantity': quantity}, + ) + + # Newly split items are recorded as created (with the consume / install event) + record_created(stock.models.StockItem, new_stock_items) + + # Flush all StockItem field changes (quantity reductions, consumption, installs) + audited_bulk_update( + stock.models.StockItem, + list(seen_stock_items.values()), + ['quantity', 'consumed_by', 'location', 'belongs_to'], ) - # Flush all StockItem field changes (quantity reductions, consumption, installs) - stock.models.StockItem.objects.bulk_update( - seen_stock_items.values(), - ['quantity', 'consumed_by', 'location', 'belongs_to'], - ) - - stock.models.StockItemTracking.objects.bulk_create(tracking_entries) - # Update build lines for "seen_build_lines" BuildLine.objects.bulk_update(seen_build_lines.values(), ['consumed']) @@ -1181,12 +1182,22 @@ class Build( if serials: """Create multiple build outputs with a single quantity of 1.""" - # Create tracking entries for each item - tracking = [] allocations = [] + # Record a tracking event for each new output outputs = stock.models.StockItem._create_serial_numbers( serials, + tracking=lambda output: { + 'entry_type': StockHistoryCode.BUILD_OUTPUT_CREATED, + 'user': user, + 'deltas': { + 'quantity': 1, + 'buildorder': self, + 'batch': output.batch, + 'serial': output.serial, + 'location': location, + }, + }, part=self.part, build=self, batch=batch, @@ -1195,21 +1206,6 @@ class Build( ) for output in outputs: - # Generate a new historical tracking entry - if entry := output.add_tracking_entry( - StockHistoryCode.BUILD_OUTPUT_CREATED, - user, - deltas={ - 'quantity': 1, - 'buildorder': self.pk, - 'batch': output.batch, - 'serial': output.serial, - 'location': location.pk if location else None, - }, - commit=False, - ): - tracking.append(entry) - # Auto-allocate stock based on serial number if auto_allocate: if new_allocations := self.auto_allocate_tracked_output( @@ -1217,16 +1213,13 @@ class Build( ): allocations.extend(new_allocations) - # Bulk create tracking entries - stock.models.StockItemTracking.objects.bulk_create(tracking, batch_size=250) - # Generate stock allocations BuildItem.objects.bulk_create(allocations, batch_size=250) else: """Create a single build output of the given quantity.""" - output = stock.models.StockItem.objects.create( + output = stock.models.StockItem( quantity=quantity, location=location, part=self.part, @@ -1235,16 +1228,17 @@ class Build( is_building=True, ) - output.add_tracking_entry( + with output.tracking_event( StockHistoryCode.BUILD_OUTPUT_CREATED, user, deltas={ 'quantity': float(quantity), - 'buildorder': self.pk, + 'buildorder': self, 'batch': batch, - 'location': location.pk if location else None, + 'location': location, }, - ) + ): + output.save() # Ensure we return a QuerySet object here, too outputs = stock.models.StockItem.objects.filter(pk=output.pk) @@ -1384,32 +1378,31 @@ class Build( ) output.build = self - # Update build output item - output.is_building = False - output.status = StockStatus.REJECTED.value - output.location = location - output.save(add_note=False) - - allocated_items = output.items_to_install.all() - - # Complete or discard allocations - if not discard_allocations: - self.complete_allocations(build_items=allocated_items, user=user) - - # Delete allocations - allocated_items.all().delete() - - output.add_tracking_entry( + with output.tracking_event( StockHistoryCode.BUILD_OUTPUT_REJECTED, user, notes=notes, deltas={ 'quantity': float(quantity), - 'location': location.pk, + 'location': location, 'status': StockStatus.REJECTED.value, - 'buildorder': self.pk, + 'buildorder': self, }, - ) + ): + # Update build output item + output.is_building = False + output.status = StockStatus.REJECTED.value + output.location = location + output.save(add_note=False) + + allocated_items = output.items_to_install.all() + + # Complete or discard allocations + if not discard_allocations: + self.complete_allocations(build_items=allocated_items, user=user) + + # Delete allocations + allocated_items.all().delete() def can_complete_output( self, @@ -1527,20 +1520,19 @@ class Build( # Assign the stock status output.set_status(status) - output.save(add_note=False) - deltas = { 'status': status, - 'buildorder': self.pk, + 'buildorder': self, 'quantity': float(output.quantity), } if location: - deltas['location'] = location.pk + deltas['location'] = location - output.add_tracking_entry( + with output.tracking_event( StockHistoryCode.BUILD_OUTPUT_COMPLETED, user, notes=notes, deltas=deltas - ) + ): + output.save(add_note=False) trigger_event(BuildEvents.OUTPUT_COMPLETED, id=output.pk, build_id=self.pk) diff --git a/src/backend/InvenTree/build/serializers.py b/src/backend/InvenTree/build/serializers.py index 7f7708c09f..6d5804aae1 100644 --- a/src/backend/InvenTree/build/serializers.py +++ b/src/backend/InvenTree/build/serializers.py @@ -536,7 +536,7 @@ class BuildOutputScrapSerializer(serializers.Serializer): notes = serializers.CharField( label=_('Notes'), - max_length=stock_models.StockItemTracking.NOTES_MAX_LENGTH, + max_length=stock_models.StockItem.TRACKING_NOTES_MAX_LENGTH, help_text=_('Reason for scrapping build output(s)'), required=True, allow_blank=False, @@ -599,7 +599,7 @@ class BuildOutputCompleteSerializer(serializers.Serializer): notes = serializers.CharField( label=_('Notes'), - max_length=stock_models.StockItemTracking.NOTES_MAX_LENGTH, + max_length=stock_models.StockItem.TRACKING_NOTES_MAX_LENGTH, required=False, allow_blank=True, ) @@ -1786,7 +1786,7 @@ class BuildConsumeSerializer(serializers.Serializer): notes = serializers.CharField( label=_('Notes'), - max_length=stock_models.StockItemTracking.NOTES_MAX_LENGTH, + max_length=stock_models.StockItem.TRACKING_NOTES_MAX_LENGTH, help_text=_('Optional notes for the stock consumption'), required=False, allow_blank=True, diff --git a/src/backend/InvenTree/build/test_api.py b/src/backend/InvenTree/build/test_api.py index 9d954a4d1b..be97907e83 100644 --- a/src/backend/InvenTree/build/test_api.py +++ b/src/backend/InvenTree/build/test_api.py @@ -11,14 +11,20 @@ from django.urls import reverse from django_q.models import OrmQ from rest_framework import status +from audit.models import AuditLogEntry from build.models import Build, BuildItem, BuildLine from build.status_codes import BuildStatus from common.models import Note from common.settings import set_global_setting -from InvenTree.unit_test import InvenTreeAPITestCase +from InvenTree.unit_test import ( + InvenTreeAPITestCase, + stock_events, + stock_history, + stock_history_count, +) from part.models import BomItem, BomItemSubstitute, Part, PartTestTemplate from stock.events import StockEvents -from stock.models import StockItem, StockItemTracking, StockLocation, StockSortOrder +from stock.models import StockItem, StockLocation, StockSortOrder from stock.status_codes import StockHistoryCode, StockStatus @@ -2427,13 +2433,13 @@ class BuildConsumeTest(BuildAPITest): consume_build_stock(self.build.pk, items=items, expected=expected) - n_tracking = StockItemTracking.objects.count() + n_tracking = stock_history_count() n_stock = StockItem.objects.count() # Simulate redelivery of the exact same task payload consume_build_stock(self.build.pk, items=items, expected=expected) - self.assertEqual(StockItemTracking.objects.count(), n_tracking) + self.assertEqual(stock_history_count(), n_tracking) self.assertEqual(StockItem.objects.count(), n_stock) self.assertEqual(self.build.consumed_stock.count(), 3) @@ -2468,12 +2474,12 @@ class BuildConsumeTest(BuildAPITest): remaining = self.build.allocated_stock.get() self.assertEqual(remaining.quantity, 60) - n_tracking = StockItemTracking.objects.count() + n_tracking = stock_history_count() # Simulate redelivery of the exact same task payload consume_build_stock(self.build.pk, lines=lines, expected=expected) - self.assertEqual(StockItemTracking.objects.count(), n_tracking) + self.assertEqual(stock_history_count(), n_tracking) remaining.refresh_from_db() self.assertEqual(remaining.quantity, 60) @@ -2749,16 +2755,15 @@ class BuildConsumeTest(BuildAPITest): self.assertEqual(si.consumed_by, build) self.assertFalse( - StockItemTracking.objects.filter( - item=si, - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value, + stock_events( + si, event_type=StockHistoryCode.SPLIT_CHILD_ITEM ).exists() ) self.assertTrue( - StockItemTracking.objects.filter( - item=si, - tracking_type=StockHistoryCode.BUILD_CONSUMED.value, - deltas__buildorder=build.pk, + stock_events( + si, + event_type=StockHistoryCode.BUILD_CONSUMED, + additional_data__deltas__buildorder__pk=build.pk, ).exists() ) @@ -2786,18 +2791,15 @@ class BuildConsumeTest(BuildAPITest): self.assertEqual(consumed_item.quantity, required_quantity) self.assertNotEqual(consumed_item.pk, stock_item.pk) - # Split tracking entries: one on the new (split-off) item, one on the original + # The new (split-off) item records its creation (as a child of the original) + created = stock_history(consumed_item).first() + self.assertEqual(created.action, AuditLogEntry.Action.CREATE) + self.assertEqual(created.changes_dict['parent'], [None, stock_item.pk]) + + # The original item records the split self.assertTrue( - StockItemTracking.objects.filter( - item=consumed_item, - tracking_type=StockHistoryCode.SPLIT_FROM_PARENT.value, - deltas__stockitem=stock_item.pk, - ).exists() - ) - self.assertTrue( - StockItemTracking.objects.filter( - item=stock_item, - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value, + stock_events( + stock_item, event_type=StockHistoryCode.SPLIT_CHILD_ITEM ).exists() ) @@ -2813,10 +2815,10 @@ class BuildConsumeTest(BuildAPITest): # Consumption is recorded against the new (split-off) item self.assertTrue( - StockItemTracking.objects.filter( - item=consumed_item, - tracking_type=StockHistoryCode.BUILD_CONSUMED.value, - deltas__buildorder=build.pk, + stock_events( + consumed_item, + event_type=StockHistoryCode.BUILD_CONSUMED, + additional_data__deltas__buildorder__pk=build.pk, ).exists() ) else: @@ -2826,16 +2828,15 @@ class BuildConsumeTest(BuildAPITest): # No split occurred - consumption is recorded directly against this item self.assertFalse( - StockItemTracking.objects.filter( - item=stock_item, - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value, + stock_events( + stock_item, event_type=StockHistoryCode.SPLIT_CHILD_ITEM ).exists() ) self.assertTrue( - StockItemTracking.objects.filter( - item=stock_item, - tracking_type=StockHistoryCode.BUILD_CONSUMED.value, - deltas__buildorder=build.pk, + stock_events( + stock_item, + event_type=StockHistoryCode.BUILD_CONSUMED, + additional_data__deltas__buildorder__pk=build.pk, ).exists() ) @@ -2951,17 +2952,17 @@ class BuildConsumeTest(BuildAPITest): ) self.assertTrue( - StockItemTracking.objects.filter( - item=component_item, - tracking_type=StockHistoryCode.INSTALLED_INTO_ASSEMBLY.value, - deltas__stockitem=output_item.pk, + stock_events( + component_item, + event_type=StockHistoryCode.INSTALLED_INTO_ASSEMBLY, + additional_data__deltas__stockitem__pk=output_item.pk, ).exists() ) self.assertTrue( - StockItemTracking.objects.filter( - item=output_item, - tracking_type=StockHistoryCode.INSTALLED_CHILD_ITEM.value, - deltas__stockitem=component_item.pk, + stock_events( + output_item, + event_type=StockHistoryCode.INSTALLED_CHILD_ITEM, + additional_data__deltas__stockitem__pk=component_item.pk, ).exists() ) diff --git a/src/backend/InvenTree/build/test_audit.py b/src/backend/InvenTree/build/test_audit.py index d44ed1f269..d9d93df902 100644 --- a/src/backend/InvenTree/build/test_audit.py +++ b/src/backend/InvenTree/build/test_audit.py @@ -152,3 +152,225 @@ class BuildAuditAPITest(InvenTreeAPITestCase): self.assertEqual(len(history), 1) self.assertEqual(history[0]['model_label'], 'Build Order') + + +class BuildStockAuditTest(BuildTestBase): + """Stock operations performed by a build order record a single audit entry per stock item.""" + + def stock_entries(self, item, n=0): + """Return the audit entries for a stock item (after the first n entries).""" + from stock.models import StockItem + + return list( + AuditLogEntry.objects.filter( + content_type=StockItem.get_content_type(), object_pk=str(item.pk) + ).order_by('pk')[n:] + ) + + def test_create_output(self): + """Creating build outputs records a single CREATE entry for each output.""" + from part.models import Part + from stock.status_codes import StockHistoryCode + + # An assembly without any trackable components + assembly = Part.objects.create( + name='Simple Assembly', description='An assembly', assembly=True + ) + build = Build.objects.create(part=assembly, quantity=5, reference='BO-9876') + + outputs = build.create_build_output(5, user=self.user) + + for output in outputs: + [entry] = self.stock_entries(output) + + self.assertEqual(entry.action, ACTION.CREATE) + self.assertEqual(entry.event_type, StockHistoryCode.BUILD_OUTPUT_CREATED) + self.assertEqual(entry.actor, self.user) + self.assertEqual( + entry.additional_data['deltas']['buildorder']['pk'], build.pk + ) + + def test_create_serialized_outputs(self): + """Creating serialized build outputs records a single CREATE entry for each output.""" + from stock.status_codes import StockHistoryCode + + self.build.part.trackable = True + self.build.part.save() + + outputs = self.build.create_build_output( + 3, serials=['1001', '1002', '1003'], user=self.user + ) + + self.assertEqual(outputs.count(), 3) + + for output in outputs: + [entry] = self.stock_entries(output) + + self.assertEqual(entry.event_type, StockHistoryCode.BUILD_OUTPUT_CREATED) + self.assertEqual(entry.changes_dict['serial'][1], output.serial) + + def test_consume(self): + """Consuming allocated stock records a single entry for each affected item.""" + from build.models import BuildItem + from stock.models import StockItem + from stock.status_codes import StockHistoryCode + + BuildItem.objects.create( + build_line=self.line_1, stock_item=self.stock_1_1, quantity=3 + ) + BuildItem.objects.create( + build_line=self.line_1, stock_item=self.stock_1_2, quantity=10 + ) + + n_1 = len(self.stock_entries(self.stock_1_1)) + n_2 = len(self.stock_entries(self.stock_1_2)) + + self.build.complete_allocations( + BuildItem.objects.filter(build_line__build=self.build), user=self.user + ) + + # Fully consumed item + [entry] = self.stock_entries(self.stock_1_1, n_1) + + self.assertEqual(entry.event_type, StockHistoryCode.BUILD_CONSUMED) + self.assertEqual(entry.changes_dict['consumed_by'], [None, self.build.pk]) + self.assertEqual(entry.actor, self.user) + + # Partially consumed item (split) + [entry] = self.stock_entries(self.stock_1_2, n_2) + + self.assertEqual(entry.event_type, StockHistoryCode.SPLIT_CHILD_ITEM) + self.assertEqual(entry.changes_dict['quantity'], ['100.00000', '90.00000']) + + # The split item is created (and consumed) + split = StockItem.objects.get(parent=self.stock_1_2) + + [entry] = self.stock_entries(split) + + self.assertEqual(entry.action, ACTION.CREATE) + self.assertEqual(entry.event_type, StockHistoryCode.BUILD_CONSUMED) + self.assertEqual(entry.changes_dict['consumed_by'][1], self.build.pk) + + def test_complete_output(self): + """Completing a build output records a single entry.""" + from stock.status_codes import StockHistoryCode + + self.build.issue_build() + + n = len(self.stock_entries(self.output_1)) + + self.build.complete_build_output(self.output_1, self.user) + + [entry] = self.stock_entries(self.output_1, n) + + self.assertEqual(entry.event_type, StockHistoryCode.BUILD_OUTPUT_COMPLETED) + self.assertEqual(entry.changes_dict['is_building'], [True, False]) + + def test_scrap_output(self): + """Scrapping a build output records a single entry.""" + from stock.models import StockLocation + from stock.status_codes import StockHistoryCode, StockStatus + + self.build.issue_build() + + location = StockLocation.objects.create(name='Scrap') + + n = len(self.stock_entries(self.output_1)) + + self.build.scrap_build_output( + self.output_1, + self.output_1.quantity, + location, + user=self.user, + notes='Scrapped', + discard_allocations=True, + ) + + [entry] = self.stock_entries(self.output_1, n) + + self.assertEqual(entry.event_type, StockHistoryCode.BUILD_OUTPUT_REJECTED) + self.assertEqual(entry.notes, 'Scrapped') + self.assertEqual( + entry.changes_dict['status'], + [StockStatus.OK.value, StockStatus.REJECTED.value], + ) + + def test_consume_install(self): + """Consuming trackable stock (installed into an output) records an entry for both items.""" + from build.models import BuildItem + from stock.models import StockItem + from stock.status_codes import StockHistoryCode + + component = StockItem.objects.create( + part=self.sub_part_3, quantity=1, serial='SN-1' + ) + + BuildItem.objects.create( + build_line=self.line_3, + stock_item=component, + quantity=1, + install_into=self.output_1, + ) + + n_component = len(self.stock_entries(component)) + n_output = len(self.stock_entries(self.output_1)) + + self.build.complete_allocations( + BuildItem.objects.filter(stock_item=component), user=self.user + ) + + [entry] = self.stock_entries(component, n_component) + + self.assertEqual(entry.event_type, StockHistoryCode.INSTALLED_INTO_ASSEMBLY) + self.assertEqual(entry.changes_dict['belongs_to'], [None, self.output_1.pk]) + self.assertEqual( + entry.additional_data['deltas']['buildorder']['pk'], self.build.pk + ) + + # Nothing changed for the output itself - the event is recorded regardless + [entry] = self.stock_entries(self.output_1, n_output) + + self.assertEqual(entry.event_type, StockHistoryCode.INSTALLED_CHILD_ITEM) + self.assertEqual(entry.changes_dict, {}) + self.assertEqual( + entry.additional_data['deltas']['stockitem']['pk'], component.pk + ) + + def test_consume_install_multiple(self): + """Installing multiple components into one output records an entry for each.""" + from build.models import BuildItem + from stock.models import StockItem + from stock.status_codes import StockHistoryCode + + components = [ + StockItem.objects.create(part=self.sub_part_3, quantity=1, serial=serial) + for serial in ['SN-1', 'SN-2'] + ] + + for component in components: + BuildItem.objects.create( + build_line=self.line_3, + stock_item=component, + quantity=1, + install_into=self.output_1, + ) + + n_output = len(self.stock_entries(self.output_1)) + + self.build.complete_allocations( + BuildItem.objects.filter(stock_item__in=components), user=self.user + ) + + entries = self.stock_entries(self.output_1, n_output) + + self.assertEqual(len(entries), 2) + + for entry in entries: + self.assertEqual(entry.event_type, StockHistoryCode.INSTALLED_CHILD_ITEM) + + self.assertEqual( + sorted( + entry.additional_data['deltas']['stockitem']['pk'] for entry in entries + ), + sorted(component.pk for component in components), + ) diff --git a/src/backend/InvenTree/build/test_build.py b/src/backend/InvenTree/build/test_build.py index b1e84e1e58..5c9091f9f5 100644 --- a/src/backend/InvenTree/build/test_build.py +++ b/src/backend/InvenTree/build/test_build.py @@ -31,15 +31,11 @@ from InvenTree.unit_test import ( InvenTreeTestCase, findOffloadedEvent, findOffloadedTask, + stock_history_count, ) from order.models import PurchaseOrder, PurchaseOrderLineItem from part.models import BomItem, BomItemSubstitute, Part, PartTestTemplate -from stock.models import ( - StockItem, - StockItemTestResult, - StockItemTracking, - StockLocation, -) +from stock.models import StockItem, StockItemTestResult, StockLocation from stock.status_codes import StockStatus from users.models import Owner @@ -1540,7 +1536,7 @@ class BuildTaskTests(BuildTestBase): self.assertEqual(self.build.status, BuildStatus.COMPLETE) n_consumed = StockItem.objects.filter(consumed_by=self.build).count() - n_tracking = StockItemTracking.objects.count() + n_tracking = stock_history_count() # A redelivered task run must skip based on the (locked) database state with mock.patch('build.tasks.trigger_event') as trigger: @@ -1550,7 +1546,7 @@ class BuildTaskTests(BuildTestBase): self.assertEqual( StockItem.objects.filter(consumed_by=self.build).count(), n_consumed ) - self.assertEqual(StockItemTracking.objects.count(), n_tracking) + self.assertEqual(stock_history_count(), n_tracking) def test_allocate_stock_merges_quantities(self): """Repeated allocations against the same (line, stock item) accumulate. @@ -1615,7 +1611,7 @@ class BuildTaskTests(BuildTestBase): self.assertEqual(self.build.completed, 10) self.assertFalse(self.output_1.is_building) - n_tracking = StockItemTracking.objects.count() + n_tracking = stock_history_count() # Run the task again (simulating a duplicated / redelivered task) build.tasks.complete_build_outputs( @@ -1630,7 +1626,7 @@ class BuildTaskTests(BuildTestBase): # and no additional stock history has been generated self.build.refresh_from_db() self.assertEqual(self.build.completed, 10) - self.assertEqual(StockItemTracking.objects.count(), n_tracking) + self.assertEqual(stock_history_count(), n_tracking) def test_complete_output_twice_rejected(self): """Completing or scrapping an already-completed output must be rejected. diff --git a/src/backend/InvenTree/common/setting/system.py b/src/backend/InvenTree/common/setting/system.py index 30c39d68fe..a14aad8c34 100644 --- a/src/backend/InvenTree/common/setting/system.py +++ b/src/backend/InvenTree/common/setting/system.py @@ -1317,23 +1317,6 @@ SYSTEM_SETTINGS: dict[str, InvenTreeSettingsKeyType] = { 'units': _('days'), 'validator': [int, MinValueValidator(30)], }, - 'STOCK_TRACKING_DELETE_OLD_ENTRIES': { - 'name': _('Delete Old Stock Tracking Entries'), - 'description': _( - 'Delete stock tracking entries older than the specified number of days' - ), - 'default': False, - 'validator': bool, - }, - 'STOCK_TRACKING_DELETE_DAYS': { - 'name': _('Stock Tracking Deletion Interval'), - 'description': _( - 'Stock tracking entries will be deleted after specified number of days' - ), - 'default': 365, - 'units': _('days'), - 'validator': [int, MinValueValidator(30)], - }, 'AUDIT_LOG_ENABLED': { 'name': _('Enable Audit Log'), 'description': _('Record a history of changes made to supported models'), diff --git a/src/backend/InvenTree/common/setting/user.py b/src/backend/InvenTree/common/setting/user.py index 2cf5c2bff0..f50c4a6ed5 100644 --- a/src/backend/InvenTree/common/setting/user.py +++ b/src/backend/InvenTree/common/setting/user.py @@ -252,7 +252,7 @@ USER_SETTINGS: dict[str, InvenTreeSettingsKeyType] = { 'DISPLAY_HISTORY_TAB': { 'name': _('Show Change History'), 'description': _( - 'Display the change history (audit log) panel on detail pages (staff users only)' + 'Display the change history (audit log) panel on detail pages' ), 'default': False, 'validator': bool, diff --git a/src/backend/InvenTree/order/models.py b/src/backend/InvenTree/order/models.py index 056adf2daa..cdf0abc873 100644 --- a/src/backend/InvenTree/order/models.py +++ b/src/backend/InvenTree/order/models.py @@ -1,6 +1,7 @@ """Order model definitions.""" import copy +from contextlib import ExitStack from decimal import Decimal from typing import Any, Optional, TypedDict @@ -32,7 +33,13 @@ import order.validators import report.mixins import stock.models import users.models as UserModels -from audit.helpers import audited_bulk_update +from audit.helpers import ( + audit_event, + audited_bulk_update, + copy_related_cache, + record_created, + write_batched, +) from build.status_codes import BuildStatus from common.currency import currency_code_default from common.notifications import InvenTreeNotificationBodies @@ -1139,22 +1146,16 @@ class PurchaseOrder(AuditedOrderMixin, TotalPriceMixin, Order): # List of stock items which have been created stock_items: list[stock.models.StockItem] = [] - # Per-item 'note' text, index-aligned with stock_items - StockItem no longer - # has its own 'notes' field, so this is threaded through to each item's - # RECEIVED_AGAINST_PURCHASE_ORDER tracking entry instead (see below) - stock_item_notes: list[str] = [] - # List of stock items to bulk create bulk_create_items: list[stock.models.StockItem] = [] # Notes for bulk_create_items, appended in lockstep - bulk_create_and_fetch() # re-fetches fresh instances from the database, so any note has to be tracked - # positionally here rather than stashed on the (discarded) unsaved instance + # positionally here rather than stashed on the (discarded) unsaved instance. + # StockItem has no 'notes' field, so the note is recorded against each item's + # RECEIVED_AGAINST_PURCHASE_ORDER tracking event instead bulk_create_notes: list[str] = [] - # List of tracking entries to create - tracking_entries: list[stock.models.StockItemTracking] = [] - # List of line items to update line_items_to_update: list[PurchaseOrderLineItem] = [] @@ -1326,14 +1327,22 @@ class PurchaseOrder(AuditedOrderMixin, TotalPriceMixin, Order): # Now, create the new stock items if serialize: + # Resolve the (possibly custom) status *before* the items are created, + # so that each new item is created (and recorded) with the selected status + status_item = stock.models.StockItem() + status_item.set_status(status, custom_values=custom_stock_status_values) + new_items = stock.models.StockItem._create_serial_numbers( - serials=serials, **stock_data + serials=serials, + tracking=lambda item, note=note: self._receive_tracking_event( + item, user, note + ), + status=status_item.status, + status_custom_key=status_item.status_custom_key, + **stock_data, ) for new_item in new_items: - new_item.set_status( - status, custom_values=custom_stock_status_values - ) # run validation for serialized items plugin.validate_batch_code new_item.validate_batch_code() @@ -1341,7 +1350,6 @@ class PurchaseOrder(AuditedOrderMixin, TotalPriceMixin, Order): new_item.run_plugin_validation() stock_items.append(new_item) - stock_item_notes.append(note) else: new_item = stock.models.StockItem(**stock_data, serial='', parent=None) @@ -1368,29 +1376,23 @@ class PurchaseOrder(AuditedOrderMixin, TotalPriceMixin, Order): bulk_create_and_fetch(stock.models.StockItem, bulk_create_items) ) - stock_items.extend(new_items) - stock_item_notes.extend(bulk_create_notes) + # Retain the cached related objects (avoid per-item lookups) + for original, new_item in zip(bulk_create_items, new_items, strict=True): + copy_related_cache(original, new_item) - # Generate a new tracking entry for each stock item - for item, item_note in zip(stock_items, stock_item_notes, strict=True): - tracking_entries.append( - item.add_tracking_entry( - StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER, - user, - deltas={ - 'location': item.location.pk if item.location else None, - 'purchaseorder': self.pk, - 'quantity': float(item.quantity), - }, - notes=item_note, - commit=False, - ) + # Record a tracking event for each new item + item_notes = dict( + zip([item.pk for item in new_items], bulk_create_notes, strict=True) ) - # Bulk create new tracking entries for each item - stock.models.StockItemTracking.objects.bulk_create( - tracking_entries, batch_size=250 - ) + stock.models.StockItem._record_created_items( + new_items, + tracking=lambda item: self._receive_tracking_event( + item, user, item_notes[item.pk] + ), + ) + + stock_items.extend(new_items) # Update received quantity for each line item audited_bulk_update(PurchaseOrderLineItem, line_items_to_update, ['received']) @@ -1425,6 +1427,19 @@ class PurchaseOrder(AuditedOrderMixin, TotalPriceMixin, Order): pk__in=[item.pk for item in stock_items] ) + def _receive_tracking_event(self, item, user, notes: str) -> dict: + """Return the tracking event for a stock item received against this order.""" + return { + 'entry_type': StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER, + 'user': user, + 'notes': notes, + 'deltas': { + 'location': item.location, + 'purchaseorder': self, + 'quantity': float(item.quantity), + }, + } + @transaction.atomic def receive_line_item( self, line, location, quantity, user, status=StockStatus.OK.value, **kwargs @@ -2974,7 +2989,7 @@ class SalesOrderShipment( Notes: This unrolls what would otherwise be a per-allocation call, so that the underlying - StockItem, StockItemTracking, SalesOrderLineItem and SalesOrderAllocation writes can + StockItem, audit log, SalesOrderLineItem and SalesOrderAllocation writes can be batched into a handful of bulk queries instead of several per allocation. """ import part.tasks @@ -3057,6 +3072,10 @@ class SalesOrderShipment( for i, (source, placeholder, quantity) in enumerate(split_items): persisted = new_stock_items[i] + + # Retain the cached related objects (avoid per-item lookups) + copy_related_cache(source, persisted) + split_item_map[id(placeholder)] = persisted split_items[i] = (source, persisted, quantity) @@ -3068,69 +3087,53 @@ class SalesOrderShipment( for allocation in allocations_to_update: allocation.item = split_item_map[id(allocation.item)] - tracking_entries = [] - split_events = [] - customer_events = [] + split_events = [ + ((), {'id': new_item.pk, 'parent': source_item.pk}) + for source_item, new_item, _quantity in split_items + ] - # Split stock items for "split_items" - for source_item, new_item, quantity in split_items: - tracking_entries.append( - stock.models.StockItemTracking( - item_id=new_item.pk, - part_id=new_item.part_id, - tracking_type=StockHistoryCode.SPLIT_FROM_PARENT.value, - user=user, - deltas={'stockitem': source_item.pk, 'quantity': float(quantity)}, - ) - ) - tracking_entries.append( - stock.models.StockItemTracking( - item_id=source_item.pk, - part_id=source_item.part_id, - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value, - user=user, - deltas={ - 'removed': float(quantity), - 'quantity': float(source_item.quantity), + customer_events = [ + ((), {'id': target_item.pk, 'customer': customer.pk if customer else None}) + for target_item, _quantity in shipped_items + ] + + # Record the operation in the audit log - a single entry for each stock item + with write_batched(), ExitStack() as events: + + def audit(item, code, deltas): + """Record a named event against the provided stock item.""" + events.enter_context(audit_event(item, code, deltas=deltas, actor=user)) + + for source_item, new_item, quantity in split_items: + audit( + source_item, + StockHistoryCode.SPLIT_CHILD_ITEM, + { + 'stockitem': new_item, + 'removed': quantity, + 'quantity': source_item.quantity, }, ) - ) - split_events.append(((), {'id': new_item.pk, 'parent': source_item.pk})) - - # Ship stock items for "shipped_items" - for target_item, quantity in shipped_items: - deltas = {'quantity': float(quantity), 'salesorder': order.pk} - - if customer is not None: - deltas['customer'] = customer.pk - deltas['customer_name'] = customer.name - - tracking_entries.append( - stock.models.StockItemTracking( - item_id=target_item.pk, - part_id=target_item.part_id, - tracking_type=StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER.value, - user=user, - deltas=deltas, + for target_item, quantity in shipped_items: + audit( + target_item, + StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER, + {'quantity': quantity, 'salesorder': order, 'customer': customer}, ) + + # Newly split items are recorded as created (with the shipment event) + record_created(stock.models.StockItem, new_stock_items) + + # Flush all StockItem field changes (quantity reductions, and shipment details) + # Note: Newly split items were created with their final state (no update required), + # and any other shipped item is one of the (existing) items drawn from + audited_bulk_update( + stock.models.StockItem, + list(seen_stock_items.values()), + ['quantity', 'sales_order', 'customer', 'location'], ) - customer_events.append(( - (), - {'id': target_item.pk, 'customer': customer.pk if customer else None}, - )) - - # Flush all StockItem field changes (quantity reductions, and shipment details) - stock.models.StockItem.objects.bulk_update( - seen_stock_items.values(), ['quantity'] - ) - stock.models.StockItem.objects.bulk_update( - [item for item, _ in shipped_items], ['sales_order', 'customer', 'location'] - ) - - stock.models.StockItemTracking.objects.bulk_create(tracking_entries) - # Update sales order lines for "seen_lines" audited_bulk_update(SalesOrderLineItem, list(seen_lines.values()), ['shipped']) @@ -3658,31 +3661,29 @@ class ReturnOrder(AuditedOrderMixin, TotalPriceMixin, Order): deltas = { 'status': status, - 'returnorder': self.pk, - 'location': location.pk, + 'returnorder': self, + 'location': location, 'quantity': float(line.quantity), } if stock_item.customer: - deltas['customer'] = stock_item.customer.pk + deltas['customer'] = stock_item.customer # Update the StockItem stock_item.set_status(status) stock_item.location = location stock_item.customer = None stock_item.sales_order = None - stock_item.save(add_note=False) - stock_item.clearAllocations() - # Add a tracking entry to the StockItem - stock_item.add_tracking_entry( + with stock_item.tracking_event( StockHistoryCode.RETURNED_AGAINST_RETURN_ORDER, user, notes=kwargs.get('note', ''), deltas=deltas, - location=location, - returnorder=self, - ) + ): + stock_item.save(add_note=False) + + stock_item.clearAllocations() # Update the LineItem line.received_date = InvenTree.helpers.current_date() diff --git a/src/backend/InvenTree/order/serializers.py b/src/backend/InvenTree/order/serializers.py index ec77957283..028daead69 100644 --- a/src/backend/InvenTree/order/serializers.py +++ b/src/backend/InvenTree/order/serializers.py @@ -19,6 +19,7 @@ import order.models import part.filters as part_filters import stock.models import stock.serializers +from audit.helpers import write_batched from common.settings import get_global_setting from company.serializers import ( AddressBriefSerializer, @@ -53,7 +54,6 @@ from order.status_codes import ( ) from part.serializers import PartBriefSerializer from plugin.base.event.events import batch_events -from stock.models import batch_tracking_entries from stock.status_codes import StockStatus from users.serializers import OwnerSerializer, UserSerializer @@ -927,7 +927,7 @@ class PurchaseOrderLineItemReceiveSerializer(serializers.Serializer): ) note = serializers.CharField( - max_length=stock.models.StockItemTracking.NOTES_MAX_LENGTH, + max_length=stock.models.StockItem.TRACKING_NOTES_MAX_LENGTH, label=_('Note'), help_text=_('Additional note for incoming stock items'), required=False, @@ -1127,7 +1127,7 @@ class PurchaseOrderReceiveSerializer(serializers.Serializer): with ( transaction.atomic(), batch_events(), - batch_tracking_entries(), + write_batched(), batch_offload_tasks(), ): items = order.receive_line_items( @@ -2344,7 +2344,7 @@ class ReturnOrderReceiveSerializer(serializers.Serializer): ) note = serializers.CharField( - max_length=stock.models.StockItemTracking.NOTES_MAX_LENGTH, + max_length=stock.models.StockItem.TRACKING_NOTES_MAX_LENGTH, label=_('Note'), help_text=_('Additional note for incoming stock items'), required=False, diff --git a/src/backend/InvenTree/order/test_api.py b/src/backend/InvenTree/order/test_api.py index be7519e1bf..68921e74f5 100644 --- a/src/backend/InvenTree/order/test_api.py +++ b/src/backend/InvenTree/order/test_api.py @@ -5,6 +5,7 @@ import io import json import threading from datetime import date, datetime, timedelta +from decimal import Decimal from typing import Optional from unittest import mock @@ -24,7 +25,7 @@ from common.currency import currency_codes from common.models import InvenTreeCustomUserStateModel, InvenTreeSetting from common.settings import set_global_setting from company.models import Address, Company, Contact, SupplierPart, SupplierPriceBreak -from InvenTree.unit_test import InvenTreeAPITestCase +from InvenTree.unit_test import InvenTreeAPITestCase, stock_history from order import models from order.models import SalesOrderAllocation, SalesOrderLineItem, SalesOrderShipment from order.serializers import TransferOrderSerialAllocationSerializer @@ -38,7 +39,7 @@ from order.status_codes import ( TransferOrderStatusGroups, ) from part.models import Part -from stock.models import StockItem, StockItemTracking, StockLocation, StockSortOrder +from stock.models import StockItem, StockLocation, StockSortOrder from stock.status_codes import StockHistoryCode, StockStatus from users.models import Owner @@ -1956,7 +1957,7 @@ class PurchaseOrderReceiveTest(OrderTest): 'batch_code': stock_field('batch'), 'packaging': stock_field('packaging'), 'barcode': stock_field('barcode_data'), - 'note': StockItemTracking.NOTES_MAX_LENGTH, + 'note': StockItem.TRACKING_NOTES_MAX_LENGTH, } line_data = {'line_item': 1, 'quantity': 10} @@ -2166,13 +2167,15 @@ class PurchaseOrderReceiveTest(OrderTest): self.assertEqual(item['status'], StockStatus.QUARANTINED) stock_item = StockItem.objects.get(pk=item['pk']) - # Check that the item has tracking entries - self.assertEqual(stock_item.tracking_info.count(), 1) - entry = stock_item.tracking_info.first() - self.assertEqual(entry.deltas['quantity'], stock_item.quantity) - self.assertEqual(entry.user, self.user) + # Check that the item has a single stock history entry (its creation) + self.assertEqual(stock_history(stock_item).count(), 1) + entry = stock_history(stock_item).first() self.assertEqual( - entry.tracking_type, StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER + Decimal(entry.changes_dict['quantity'][1]), stock_item.quantity + ) + self.assertEqual(entry.actor, self.user) + self.assertEqual( + entry.event_type, StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER ) # Check that the order has been completed @@ -2207,10 +2210,10 @@ class PurchaseOrderReceiveTest(OrderTest): stock_item = StockItem.objects.get(pk=response[0]['pk']) - self.assertEqual(stock_item.tracking_info.count(), 1) - entry = stock_item.tracking_info.first() + self.assertEqual(stock_history(stock_item).count(), 1) + entry = stock_history(stock_item).first() self.assertEqual( - entry.tracking_type, StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER + entry.event_type, StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER ) self.assertEqual(entry.notes, 'Damaged box, 2 units short') @@ -2239,10 +2242,86 @@ class PurchaseOrderReceiveTest(OrderTest): for i in range(200, 203): item = StockItem.objects.get(serial_int=i) - self.assertEqual(item.tracking_info.count(), 1) - entry = item.tracking_info.first() + self.assertEqual(stock_history(item).count(), 1) + entry = stock_history(item).first() + self.assertEqual( + entry.event_type, StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER + ) self.assertEqual(entry.notes, 'Received via serialized batch') + def test_receive_serialized_status(self): + """Test that the selected status (including custom status values) is applied to received serialized items. + + Regression test: the status was previously only applied to the (already created) + serialized items in memory, and was never saved to the database. + """ + self.post( + self.url, + { + 'items': [ + { + 'line_item': 1, + 'quantity': 3, + 'serial_numbers': '300+', + 'status': StockStatus.DAMAGED.value, + } + ], + 'location': 1, + }, + expected_code=201, + ) + + for i in range(300, 303): + item = StockItem.objects.get(serial_int=i) + self.assertEqual(item.status, StockStatus.DAMAGED.value) + + # The status is recorded against the creation of the item + [entry] = stock_history(item) + self.assertEqual( + entry.changes_dict['status'], [None, StockStatus.DAMAGED.value] + ) + + # A custom status value is also applied (and recorded) + custom_status = InvenTreeCustomUserStateModel.objects.create( + key=1055, + name='Damaged (cosmetic)', + label='Damaged (cosmetic)', + color='warning', + logical_key=StockStatus.DAMAGED.value, + model=ContentType.objects.get_for_model(StockItem), + reference_status='StockStatus', + ) + + self.post( + self.url, + { + 'items': [ + { + 'line_item': 1, + 'quantity': 3, + 'serial_numbers': '400+', + 'status': custom_status.key, + } + ], + 'location': 1, + }, + expected_code=201, + ) + + for i in range(400, 403): + item = StockItem.objects.get(serial_int=i) + self.assertEqual(item.status, StockStatus.DAMAGED.value) + self.assertEqual(item.status_custom_key, custom_status.key) + self.assertEqual(item.get_custom_status(), custom_status.key) + + [entry] = stock_history(item) + self.assertEqual( + entry.changes_dict['status'], [None, StockStatus.DAMAGED.value] + ) + self.assertEqual( + entry.changes_dict['status_custom_key'], [None, custom_status.key] + ) + def test_bulk_receive_query_benchmark(self): """Benchmark: measure the number of DB queries required to receive 100 line items at once.""" InvenTreeSetting.set_setting('ENABLE_PLUGINS_EVENTS', True, change_user=None) @@ -4001,7 +4080,7 @@ class ReturnOrderTests(InvenTreeAPITestCase): rma = models.ReturnOrder.objects.first() url = reverse('api-return-order-receive', kwargs={'pk': rma.pk}) - max_length = StockItemTracking.NOTES_MAX_LENGTH + max_length = StockItem.TRACKING_NOTES_MAX_LENGTH response = self.post( url, @@ -4222,7 +4301,7 @@ class ReturnOrderTests(InvenTreeAPITestCase): # Get a reference to one of the stock items stock_item = rma.lines.first().item - n_tracking = stock_item.tracking_info.count() + n_tracking = stock_history(stock_item).count() # Receive items successfully data = receive( @@ -4234,16 +4313,19 @@ class ReturnOrderTests(InvenTreeAPITestCase): self.assertTrue(line.received) self.assertIsNotNone(line.received_date) - # A single tracking entry should have been added to the item - self.assertEqual(stock_item.tracking_info.count(), n_tracking + 1) + # A single audit entry should have been added to the item + self.assertEqual(stock_history(stock_item).count(), n_tracking + 1) - tracking_entry = stock_item.tracking_info.last() - deltas = tracking_entry.deltas + entry = stock_history(stock_item).last() + self.assertEqual( + entry.event_type, StockHistoryCode.RETURNED_AGAINST_RETURN_ORDER + ) - self.assertEqual(deltas['status'], StockStatus.QUARANTINED) - self.assertEqual(deltas['customer'], customer.pk) - self.assertEqual(deltas['location'], 1) - self.assertEqual(deltas['returnorder'], rma.pk) + # Field changes are recorded as such (and not duplicated in the deltas) + self.assertEqual(entry.changes_dict['status'][1], StockStatus.QUARANTINED) + self.assertEqual(entry.changes_dict['customer'], [customer.pk, None]) + self.assertEqual(entry.changes_dict['location'][1], 1) + self.assertEqual(entry.additional_data['deltas']['returnorder']['pk'], rma.pk) def test_receive_untracked(self): """Test that we can receive untracked items against a ReturnOrder. diff --git a/src/backend/InvenTree/order/test_audit.py b/src/backend/InvenTree/order/test_audit.py index 99691e7041..70ef61c832 100644 --- a/src/backend/InvenTree/order/test_audit.py +++ b/src/backend/InvenTree/order/test_audit.py @@ -264,6 +264,23 @@ class PurchaseOrderAuditTest(OrderAuditTestBase): self.assertEqual(new, line.received + 3) self.assertEqual(entry.parent_object_pk, str(po.pk)) + # Each received stock item records a single entry (creation, with the event) + from stock.models import StockItem + from stock.status_codes import StockHistoryCode + + items = StockItem.objects.filter(purchase_order=po) + self.assertEqual(items.count(), 2) + + for item in items: + [entry] = self.entries(StockItem, item.pk) + + self.assertEqual(entry.action, ACTION.CREATE) + self.assertEqual( + entry.event_type, StockHistoryCode.RECEIVED_AGAINST_PURCHASE_ORDER + ) + self.assertEqual(entry.actor, self.user) + self.assertEqual(entry.changes_dict['location'][1], location.pk) + class SalesOrderAuditTest(OrderAuditTestBase): """Audit tests for SalesOrder, line items, extra lines and shipments.""" @@ -412,6 +429,27 @@ class SalesOrderAuditTest(OrderAuditTestBase): self.assertIsNone(shipment_entry.changes_dict['shipment_date'][0]) + # Stock items record a single entry each + from stock.models import StockItem + from stock.status_codes import StockHistoryCode + + [source_entry] = self.entries( + StockItem, self.stock.pk, event_type__isnull=False + ).exclude(event_type=StockHistoryCode.CREATED) + + self.assertEqual(source_entry.event_type, StockHistoryCode.SPLIT_CHILD_ITEM) + self.assertEqual(source_entry.actor, self.user) + + shipped = StockItem.objects.get(parent=self.stock) + + [shipped_entry] = self.entries(StockItem, shipped.pk) + + self.assertEqual(shipped_entry.action, ACTION.CREATE) + self.assertEqual( + shipped_entry.event_type, StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER + ) + self.assertEqual(shipped_entry.changes_dict['customer'][1], self.customer.pk) + class ReturnOrderAuditTest(OrderAuditTestBase): """Audit tests for ReturnOrder, line items and extra lines.""" @@ -484,6 +522,36 @@ class ReturnOrderAuditTest(OrderAuditTestBase): update.changes_dict['description'], ['Restocking fee', 'Inspection fee'] ) + def test_receive_stock(self): + """Receiving an item against a return order records a single entry.""" + from stock.models import StockItem + from stock.status_codes import StockHistoryCode, StockStatus + + location = StockLocation.objects.create(name='Returns') + + line = models.ReturnOrderLineItem.objects.create( + order=self.order, item=self.item + ) + + self.order.issue_order() + + n = self.entries(StockItem, self.item.pk).count() + + self.order.receive_line_item(line, location, self.user, note='Inspected') + + [entry] = list(self.entries(StockItem, self.item.pk).order_by('pk')[n:]) + + self.assertEqual( + entry.event_type, StockHistoryCode.RETURNED_AGAINST_RETURN_ORDER + ) + self.assertEqual(entry.actor, self.user) + self.assertEqual(entry.notes, 'Inspected') + self.assertEqual(entry.changes_dict['location'][1], location.pk) + self.assertEqual(entry.changes_dict['status'][1], StockStatus.QUARANTINED.value) + self.assertEqual( + entry.additional_data['deltas']['returnorder']['pk'], self.order.pk + ) + class TransferOrderAuditTest(OrderAuditTestBase): """Audit tests for TransferOrder and line items.""" @@ -566,3 +634,17 @@ class TransferOrderAuditTest(OrderAuditTestBase): ) self.assertNotIn('F(', str(entry.changes)) self.assertEqual(entry.parent_object_pk, str(self.order.pk)) + + # The transferred stock records a single STOCK_MOVE event (linked to the order) + from stock.models import StockItem + from stock.status_codes import StockHistoryCode + + moved = StockItem.objects.get(parent=self.stock) + + [created] = self.entries(StockItem, moved.pk) + + self.assertEqual(created.event_type, StockHistoryCode.SPLIT_FROM_PARENT) + self.assertEqual(created.changes_dict['location'][1], self.target.pk) + self.assertEqual( + created.additional_data['deltas']['transferorder']['pk'], self.order.pk + ) diff --git a/src/backend/InvenTree/order/test_sales_order.py b/src/backend/InvenTree/order/test_sales_order.py index 47ba514bc2..88d1e7ea48 100644 --- a/src/backend/InvenTree/order/test_sales_order.py +++ b/src/backend/InvenTree/order/test_sales_order.py @@ -24,6 +24,9 @@ from InvenTree.unit_test import ( InvenTreeAPITestCase, InvenTreeTestCase, addUserPermission, + stock_events, + stock_history, + stock_history_count, ) from order.models import ( SalesOrder, @@ -35,7 +38,7 @@ from order.models import ( from order.serializers import SalesOrderShipmentAllocationSerializer from part.models import Part from stock.events import StockEvents -from stock.models import StockItem, StockItemTracking, StockLocation +from stock.models import StockItem, StockLocation from stock.status_codes import StockHistoryCode from users.models import Owner @@ -506,7 +509,7 @@ class SalesOrderTest(InvenTreeAPITestCase): self.assertFalse(self.shipment.check_can_complete(raise_error=False)) n_items = StockItem.objects.count() - n_tracking = StockItemTracking.objects.count() + n_tracking = stock_history_count() # Simulate a duplicated task execution # (e.g. a second completion request submitted before the first task ran) @@ -519,7 +522,7 @@ class SalesOrderTest(InvenTreeAPITestCase): # No additional stock items or tracking entries have been created self.assertEqual(StockItem.objects.count(), n_items) - self.assertEqual(StockItemTracking.objects.count(), n_tracking) + self.assertEqual(stock_history_count(), n_tracking) def test_shipment_many_items(self): """Test completion of a shipment with many items. @@ -618,8 +621,8 @@ class SalesOrderTest(InvenTreeAPITestCase): url, expected_code=200, benchmark=True, - max_query_time=5.0, - max_query_count=120, + max_query_time=10.0, + max_query_count=150, ) self.assertEqual(response.status_code, 200) @@ -674,25 +677,18 @@ class SalesOrderTest(InvenTreeAPITestCase): self.assertEqual(len(low_stock_tasks), 1) self.assertEqual(low_stock_tasks[0].args(), (part.pk,)) - # Exactly one StockItemTracking entry should exist per split, per side + # Exactly one 'split' event should exist per split (on the original item) self.assertEqual( - StockItemTracking.objects.filter( - tracking_type=StockHistoryCode.SPLIT_FROM_PARENT.value - ).count(), - len(split_pks), - ) - self.assertEqual( - StockItemTracking.objects.filter( - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value - ).count(), + stock_history_count(event_type=StockHistoryCode.SPLIT_CHILD_ITEM), len(split_pks), ) - # Exactly one 'shipped' tracking entry should exist per allocated item + # Exactly one 'shipped' event should exist per allocated item + # (for split items, this is recorded against the creation of the new item) self.assertEqual( - StockItemTracking.objects.filter( - tracking_type=StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER.value - ).count(), + stock_history_count( + event_type=StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER + ), N_ITEMS, ) @@ -704,15 +700,15 @@ class SalesOrderTest(InvenTreeAPITestCase): self.assertEqual(item.customer, customer) self.assertFalse( - StockItemTracking.objects.filter( - item=item, tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value + stock_events( + item, event_type=StockHistoryCode.SPLIT_CHILD_ITEM ).exists() ) self.assertTrue( - StockItemTracking.objects.filter( - item=item, - tracking_type=StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER.value, - deltas__salesorder=so.pk, + stock_events( + item, + event_type=StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER, + additional_data__deltas__salesorder__pk=so.pk, ).exists() ) @@ -735,24 +731,20 @@ class SalesOrderTest(InvenTreeAPITestCase): ) self.assertEqual(sample_new_item.quantity, 1) - self.assertTrue( - StockItemTracking.objects.filter( - item=sample_new_item, - tracking_type=StockHistoryCode.SPLIT_FROM_PARENT.value, - deltas__stockitem=sample_original.pk, - ).exists() + + # The new item records a single entry - its creation (as a child of the original), + # with the 'shipped' event + [created] = stock_history(sample_new_item) + self.assertEqual(created.changes_dict['parent'], [None, sample_original.pk]) + self.assertEqual( + created.event_type, StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER ) + self.assertEqual(created.additional_data['deltas']['salesorder']['pk'], so.pk) + + # The original item records the split self.assertTrue( - StockItemTracking.objects.filter( - item=sample_original, - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value, - ).exists() - ) - self.assertTrue( - StockItemTracking.objects.filter( - item=sample_new_item, - tracking_type=StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER.value, - deltas__salesorder=so.pk, + stock_events( + sample_original, event_type=StockHistoryCode.SPLIT_CHILD_ITEM ).exists() ) self.assertTrue( diff --git a/src/backend/InvenTree/stock/__init__.py b/src/backend/InvenTree/stock/__init__.py index 524da0600c..43a6dd2853 100644 --- a/src/backend/InvenTree/stock/__init__.py +++ b/src/backend/InvenTree/stock/__init__.py @@ -4,5 +4,4 @@ It includes models for: - StockLocation - StockItem -- StockItemTracking """ diff --git a/src/backend/InvenTree/stock/admin.py b/src/backend/InvenTree/stock/admin.py index 5326f5cc33..585c523184 100644 --- a/src/backend/InvenTree/stock/admin.py +++ b/src/backend/InvenTree/stock/admin.py @@ -3,15 +3,7 @@ from django.contrib import admin from django.db.models import Count -from common.admin_shared import ReadOnlyAdminMixin - -from .models import ( - StockItem, - StockItemTestResult, - StockItemTracking, - StockLocation, - StockLocationType, -) +from .models import StockItem, StockItemTestResult, StockLocation, StockLocationType class LocationInline(admin.TabularInline): @@ -78,17 +70,6 @@ class StockItemAdmin(admin.ModelAdmin): ] -@admin.register(StockItemTracking) -class StockTrackingAdmin(ReadOnlyAdminMixin, admin.ModelAdmin): - """Admin class for StockTracking - read-only to preserve audit trail integrity.""" - - list_display = ('item', 'date', 'label') - - search_fields = ['item__part__name', 'item__serial', 'notes'] - - autocomplete_fields = ['item'] - - @admin.register(StockItemTestResult) class StockItemTestResultAdmin(admin.ModelAdmin): """Admin class for StockItemTestResult.""" diff --git a/src/backend/InvenTree/stock/api.py b/src/backend/InvenTree/stock/api.py index 7625190774..0cb6072aba 100644 --- a/src/backend/InvenTree/stock/api.py +++ b/src/backend/InvenTree/stock/api.py @@ -26,9 +26,7 @@ import InvenTree.helpers import InvenTree.permissions import stock.serializers as StockSerializers from build.models import Build -from build.serializers import BuildSerializer from company.models import Company, ManufacturerPart, SupplierPart -from company.serializers import CompanySerializer from data_exporter.mixins import DataExportViewMixin from generic.states.api import StatusView from InvenTree.api import ( @@ -56,20 +54,11 @@ from InvenTree.mixins import ( SerializerContextMixin, ) from InvenTree.serializers import apply_duplicate_copy_options -from order.models import PurchaseOrder, ReturnOrder, SalesOrder, TransferOrder -from order.serializers import ( - PurchaseOrderSerializer, - ReturnOrderSerializer, - SalesOrderSerializer, - TransferOrderSerializer, -) from part.models import BomItem, Part, PartCategory -from part.serializers import PartBriefSerializer from stock.generators import generate_batch_code, generate_serial_number from stock.models import ( StockItem, StockItemTestResult, - StockItemTracking, StockLocation, StockLocationType, ) @@ -1287,46 +1276,43 @@ class StockList( original = duplicate['original'] - # copy_history/copy_tests don't follow the copy__from() naming - # convention (copyHistoryFrom/copyTestResultsFrom), so still need + # copy_tests doesn't follow the copy__from() naming + # convention (copyTestResultsFrom), so still needs # handling here - only copy_notes can go through the shared helper apply_duplicate_copy_options(item, duplicate, original, copy_notes=True) - if duplicate.get('copy_history', False): - item.copyHistoryFrom(original) - if duplicate.get('copy_tests', False): item.copyTestResultsFrom(original) with transaction.atomic(): if serials: + create_data = {**serializer.validated_data} + + if status_value: + # Resolve the (possibly custom) status before the items are created, + # so that each item is created with the correct status + status_item = StockItem() + + if status_item.set_status(status_value): + create_data['status'] = status_item.status + create_data['status_custom_key'] = status_item.status_custom_key + # Create multiple serialized StockItem objects + # (recording a tracking event for each new item) items = StockItem._create_serial_numbers( - serials, **serializer.validated_data + serials, + tracking=lambda item: { + 'entry_type': StockHistoryCode.CREATED, + 'user': user, + 'deltas': {'status': status_value}, + 'quantity': float(item.quantity), + }, + **create_data, ) - # Next, bulk-create stock tracking entries for the newly created items - tracking = [] - for item in items: - if status_value and not item.compare_status(status_value): - item.set_status(status_value) - item.save() - apply_duplicate_options(item) - if entry := item.add_tracking_entry( - StockHistoryCode.CREATED, - user, - deltas={'status': status_value}, - location=location, - quantity=float(item.quantity), - commit=False, - ): - tracking.append(entry) - - StockItemTracking.objects.bulk_create(tracking, batch_size=250) - # Annotate the stock items with part information queryset = StockSerializers.StockItemSerializer.annotate_queryset(items) @@ -1578,161 +1564,6 @@ class StockItemTestResultList( serializer.save(user=self.request.user) -class StockTrackingDetail(RetrieveAPI): - """Detail API endpoint for StockItemTracking model.""" - - queryset = StockItemTracking.objects.all() - serializer_class = StockSerializers.StockTrackingSerializer - - -class StockTrackingOutputOptions(OutputConfiguration): - """Output options for StockItemTracking endpoint.""" - - OPTIONS = [ - InvenTreeOutputOption(flag='item_detail'), - InvenTreeOutputOption(flag='user_detail'), - ] - - -class StockTrackingFilter(FilterSet): - """API filter options for the StockTrackingList endpoint.""" - - class Meta: - """Metaclass options.""" - - model = StockItemTracking - fields = ['item', 'user'] - - include_variants = rest_filters.BooleanFilter( - label=_('Include Part Variants'), method='filter_include_variants' - ) - - def filter_include_variants(self, queryset, name, value): - """Filter by whether or not to include part variants. - - Note: - - This filter does nothing by itself, and is only used to modify the behavior of the 'part' filter. - - Refer to the 'filter_part' method for more information on how this works. - """ - return queryset - - part = rest_filters.ModelChoiceFilter( - label=_('Part'), queryset=Part.objects.all(), method='filter_part' - ) - - def filter_part(self, queryset, name, part): - """Filter StockTracking entries by the linked part. - - Note: - - This filter behavior also takes into account the 'include_variants' filter, which determines whether or not to include part variants in the results. - """ - include_variants = str2bool(self.data.get('include_variants', False)) - - if include_variants: - return queryset.filter(part__in=part.get_descendants(include_self=True)) - else: - return queryset.filter(part=part) - - min_date = InvenTreeDateFilter( - label=_('Date after'), field_name='date', lookup_expr='gt' - ) - - max_date = InvenTreeDateFilter( - label=_('Date before'), field_name='date', lookup_expr='lt' - ) - - -class StockTrackingList( - SerializerContextMixin, DataExportViewMixin, OutputOptionsMixin, ListAPI -): - """API endpoint for list view of StockItemTracking objects. - - StockItemTracking objects are read-only - (they are created by internal model functionality) - - - GET: Return list of StockItemTracking objects - """ - - queryset = StockItemTracking.objects.all().prefetch_related('item', 'part') - serializer_class = StockSerializers.StockTrackingSerializer - filterset_class = StockTrackingFilter - output_options = StockTrackingOutputOptions - - def get_delta_model_map(self) -> dict: - """Return a mapping of delta models to their respective models and serializers. - - This is used to generate additional context information for the historical data, - with some attempt at caching so that we can reduce the number of database hits. - """ - return { - 'part': (Part, PartBriefSerializer), - 'location': (StockLocation, StockSerializers.LocationSerializer), - 'customer': (Company, CompanySerializer), - 'purchaseorder': (PurchaseOrder, PurchaseOrderSerializer), - 'salesorder': (SalesOrder, SalesOrderSerializer), - 'returnorder': (ReturnOrder, ReturnOrderSerializer), - 'transferorder': (TransferOrder, TransferOrderSerializer), - 'buildorder': (Build, BuildSerializer), - 'item': (StockItem, StockSerializers.StockItemSerializer), - 'stockitem': (StockItem, StockSerializers.StockItemSerializer), - } - - def list(self, request, *args, **kwargs): - """List all stock tracking entries.""" - queryset = self.filter_queryset(self.get_queryset()) - - page = self.paginate_queryset(queryset) - - if page is not None: - serializer = self.get_serializer(page, many=True) - else: - serializer = self.get_serializer(queryset, many=True) - - data = serializer.data - - delta_models = self.get_delta_model_map() - - # Construct a set of related models we need to lookup for later - related_model_lookups = {key: set() for key in delta_models} - - # Run a first pass through the data to determine which related models we need to lookup - for item in data: - deltas = item['deltas'] or {} - - for key in delta_models: - if key in deltas: - related_model_lookups[key].add(deltas[key]) - - for key in delta_models: - model, serializer = delta_models[key] - - # Fetch all related models in one go - related_models = model.objects.filter(pk__in=related_model_lookups[key]) - - # Construct a mapping of pk -> serialized data - related_data = {obj.pk: serializer(obj).data for obj in related_models} - - # Now, update the data with the serialized data - for item in data: - deltas = item['deltas'] or {} - - if key in deltas: - item['deltas'][f'{key}_detail'] = related_data.get(deltas[key]) - - if page is not None: - return self.get_paginated_response(data) - - return Response(data) - - filter_backends = SEARCH_ORDER_FILTER - - ordering = '-date' - - ordering_fields = ['date'] - - search_fields = ['notes'] - - stock_api_urls = [ path( 'location/', @@ -1796,25 +1627,6 @@ stock_api_urls = [ ), ]), ), - # StockItemTracking API endpoints - path( - 'track/', - include([ - path( - '/', - StockTrackingDetail.as_view(), - name='api-stock-tracking-detail', - ), - # Stock tracking status code information - path( - 'status/', - StatusView.as_view(), - {StatusView.MODEL_REF: StockHistoryCode}, - name='api-stock-tracking-status-codes', - ), - path('', StockTrackingList.as_view(), name='api-stock-tracking-list'), - ]), - ), # Detail views for a single stock item path( '/', diff --git a/src/backend/InvenTree/stock/migrations/0129_migrate_stock_tracking_to_audit_log.py b/src/backend/InvenTree/stock/migrations/0129_migrate_stock_tracking_to_audit_log.py new file mode 100644 index 0000000000..452098ae48 --- /dev/null +++ b/src/backend/InvenTree/stock/migrations/0129_migrate_stock_tracking_to_audit_log.py @@ -0,0 +1,341 @@ +"""Copy legacy stock tracking entries (StockItemTracking) into the audit log (AuditLogEntry). + +Each StockItemTracking row is converted to a single AuditLogEntry: + +- The entry is recorded against the stock item (even if the item has since been deleted) +- The parent reference is the part (so the entry is included in the history of the part) +- The tracking type is recorded as the event type (StockHistoryCode values are unchanged) + - 'EDITED' entries are recorded as regular updates (no event type) + - Entries which record the creation of a stock item are recorded as CREATE entries +- The user, date and notes are copied across +- Tracking deltas are converted to audit log changes / deltas: + - Status changes are recorded as field changes ('status' / 'status_custom_key') + - Location changes (with a known previous location) are recorded as field changes + - For CREATE entries, initial field values are recorded as field changes + - References to other objects (by pk) are recorded as {model_type, pk, str} references + - Any other deltas are copied as-is +- The pk of the original tracking entry is retained in additional_data['tracking_id'] + +The legacy StockItemTracking rows are not modified (or deleted) by this migration. +""" + +from decimal import Decimal, InvalidOperation + +from django.db import migrations + +# StockHistoryCode values (copied here, as migrations should not import application code) +CREATED = 1 +EDITED = 5 +SPLIT_FROM_PARENT = 40 +CREATED_FROM_DISASSEMBLY = 47 +BUILD_OUTPUT_CREATED = 50 +RECEIVED_AGAINST_PURCHASE_ORDER = 70 + +# Tracking codes which record the creation of a stock item +CREATE_CODES = { + CREATED, + SPLIT_FROM_PARENT, + CREATED_FROM_DISASSEMBLY, + BUILD_OUTPUT_CREATED, + RECEIVED_AGAINST_PURCHASE_ORDER, +} + +# AuditLogEntry action codes (as defined by django-auditlog) +ACTION_CREATE = 0 +ACTION_UPDATE = 1 + +# Delta keys which reference other objects (by pk) +REFERENCE_KEYS = { + 'stockitem': ('stock', 'stockitem'), + 'location': ('stock', 'stocklocation'), + 'old_location': ('stock', 'stocklocation'), + 'stockitemlocation': ('stock', 'stocklocation'), + 'part': ('part', 'part'), + 'customer': ('company', 'company'), + 'buildorder': ('build', 'build'), + 'build_order': ('build', 'build'), + 'purchaseorder': ('order', 'purchaseorder'), + 'salesorder': ('order', 'salesorder'), + 'returnorder': ('order', 'returnorder'), + 'transferorder': ('order', 'transferorder'), +} + +# Delta keys which are recorded as field changes (rather than deltas) +STATUS_KEYS = ('status', 'status_logical', 'old_status', 'old_status_logical') + +# For CREATE entries, these deltas record the initial value of the stock item field +CREATE_FIELD_KEYS = ('quantity', 'location', 'serial', 'batch') + +BATCH_SIZE = 1000 + + +def format_quantity(quantity) -> str: + """Format a stock quantity for display (e.g. '10', '2.5').""" + try: + value = Decimal(str(quantity)).normalize() + except (InvalidOperation, TypeError, ValueError): + return str(quantity) + + # Remove exponent notation (e.g. 1E+1) + return format(value, 'f') + + +def stock_item_str(item) -> str: + """Return a string representation of a stock item. + + Note: This approximates StockItem.__str__ - historical models do not provide model + methods, and the configurable part name format cannot be applied here. + """ + if item.serial: + return f'{item.part.name} #{item.serial}' + + return f'{format_quantity(item.quantity)} x {item.part.name}' + + +def reference_str(model_type: str, instance) -> str: + """Return a string representation for a referenced object.""" + if model_type == 'stock.stockitem': + return stock_item_str(instance) + + if model_type == 'stock.stocklocation': + return instance.pathstring or instance.name + + if model_type in ( + 'build.build', + 'order.purchaseorder', + 'order.salesorder', + 'order.returnorder', + 'order.transferorder', + ): + return instance.reference + + return instance.name + + +def to_pk(value): + """Convert a delta value to a primary key (or None).""" + if value is None or isinstance(value, bool): + return None + + try: + return int(value) + except (TypeError, ValueError): + return None + + +def convert_entry(entry, references: dict) -> dict: + """Convert a single StockItemTracking entry into AuditLogEntry field values.""" + code = entry.tracking_type + deltas = dict(entry.deltas or {}) if isinstance(entry.deltas, dict) else {} + + action = ACTION_CREATE if code in CREATE_CODES else ACTION_UPDATE + + changes = {} + + # Status changes are recorded as field changes + if 'status' in deltas or 'status_logical' in deltas: + new_status = deltas.get('status') + new_logical = deltas.get('status_logical', new_status) + old_status = deltas.get('old_status') + old_logical = deltas.get('old_status_logical', old_status) + + changes['status'] = [old_logical, new_logical] + + # A custom status is recorded separately to the logical status + new_custom = new_status if new_status != new_logical else None + old_custom = old_status if old_status != old_logical else None + + if new_custom is not None or old_custom is not None: + changes['status_custom_key'] = [old_custom, new_custom] + + for key in STATUS_KEYS: + deltas.pop(key, None) + + # A location change (with known previous location) is recorded as a field change + if 'old_location' in deltas: + old_location = to_pk(deltas.pop('old_location')) + new_location = to_pk(deltas.pop('location', None)) + + if old_location != new_location: + changes['location'] = [old_location, new_location] + + # The initial values of a new stock item are recorded as field changes + if action == ACTION_CREATE: + for key in CREATE_FIELD_KEYS: + if key in deltas: + value = deltas.pop(key) + + if key == 'location': + value = to_pk(value) + elif key == 'quantity': + value = format_quantity(value) + elif value is not None: + value = str(value) + + if value is not None: + changes[key] = [None, value] + + # The customer name is recorded as the string representation of the customer reference + customer_name = deltas.pop('customer_name', None) + + # Convert references (by pk) to {model_type, pk, str} + for key, (app_label, model_name) in REFERENCE_KEYS.items(): + if key not in deltas: + continue + + pk = to_pk(deltas[key]) + + if pk is None: + continue + + model_type = f'{app_label}.{model_name}' + instance = references.get(model_type, {}).get(pk) + + if instance is not None: + text = reference_str(model_type, instance) + elif key == 'customer' and customer_name: + text = customer_name + else: + # The referenced object no longer exists + text = None + + deltas[key] = {'model_type': model_type, 'pk': pk, 'str': text} + + additional_data = {'tracking_id': entry.pk} + + if deltas: + additional_data['deltas'] = deltas + + item = entry.item + + if item is not None: + object_repr = stock_item_str(item) + elif entry.part is not None: + object_repr = entry.part.name + else: + object_repr = '' + + user = entry.user + + return { + 'object_pk': str(entry.item_id) if entry.item_id is not None else '', + 'object_id': entry.item_id, + 'object_repr': object_repr, + 'action': action, + 'changes': changes, + 'changes_text': '', + 'timestamp': entry.date, + 'actor_id': entry.user_id, + 'actor_email': getattr(user, 'email', None) or None, + 'actor_username': getattr(user, 'username', None), + 'parent_object_pk': str(entry.part_id) if entry.part_id is not None else None, + 'event_type': None if code == EDITED else code, + 'notes': entry.notes[:512] if entry.notes else None, + 'additional_data': additional_data, + } + + +def collect_references(apps, entries) -> dict: + """Fetch all objects referenced (by pk) in the deltas of the provided entries.""" + required = {} + + for entry in entries: + if not isinstance(entry.deltas, dict): + continue + + for key, (app_label, model_name) in REFERENCE_KEYS.items(): + if (pk := to_pk(entry.deltas.get(key))) is not None: + required.setdefault(f'{app_label}.{model_name}', set()).add(pk) + + references = {} + + for model_type, pks in required.items(): + app_label, model_name = model_type.split('.') + model = apps.get_model(app_label, model_name) + + queryset = model.objects.all() + + if model_type == 'stock.stockitem': + queryset = queryset.select_related('part') + + references[model_type] = queryset.in_bulk(list(pks)) + + return references + + +def migrate_tracking_entries(apps, schema_editor): + """Copy all StockItemTracking entries into the audit log.""" + StockItemTracking = apps.get_model('stock', 'stockitemtracking') + AuditLogEntry = apps.get_model('audit', 'auditlogentry') + ContentType = apps.get_model('contenttypes', 'contenttype') + + n = StockItemTracking.objects.count() + + if n == 0: + return + + print(f'\nstock.migrations.0129: Migrating {n} stock tracking entries to the audit log') + + stock_ct, _ = ContentType.objects.get_or_create( + app_label='stock', model='stockitem' + ) + part_ct, _ = ContentType.objects.get_or_create(app_label='part', model='part') + + queryset = StockItemTracking.objects.select_related( + 'item', 'item__part', 'part', 'user' + ).order_by('pk') + + last_pk = 0 + migrated = 0 + + while True: + entries = list(queryset.filter(pk__gt=last_pk)[:BATCH_SIZE]) + + if not entries: + break + + references = collect_references(apps, entries) + + audit_entries = [] + + for entry in entries: + data = convert_entry(entry, references) + + audit_entries.append( + AuditLogEntry( + content_type_id=stock_ct.pk, + parent_content_type_id=part_ct.pk + if data['parent_object_pk'] is not None + else None, + **data, + ) + ) + + AuditLogEntry.objects.bulk_create(audit_entries, batch_size=BATCH_SIZE) + + migrated += len(audit_entries) + last_pk = entries[-1].pk + + print(f'- Migrated {migrated} / {n} entries') + + +class Migration(migrations.Migration): + """Copy legacy stock tracking entries into the audit log.""" + + dependencies = [ + ('stock', '0128_remove_stockitem_notes'), + ('audit', '0003_event_type'), + ('build', '0060_remove_build_notes'), + ('company', '0083_company_is_internal'), + ('order', '0124_purchaseorder_delivery_address'), + ('part', '0155_move_pricing_models'), + ('contenttypes', '0002_remove_content_type_name'), + ] + + operations = [ + # The migrated entries are retained on reversal, as the legacy table + # (dropped in the following migration) cannot be restored + migrations.RunPython( + migrate_tracking_entries, reverse_code=migrations.RunPython.noop + ) + ] diff --git a/src/backend/InvenTree/stock/migrations/0130_delete_stockitemtracking.py b/src/backend/InvenTree/stock/migrations/0130_delete_stockitemtracking.py new file mode 100644 index 0000000000..14d1027968 --- /dev/null +++ b/src/backend/InvenTree/stock/migrations/0130_delete_stockitemtracking.py @@ -0,0 +1,9 @@ +# Generated by Django 5.2.17 on 2026-10-25 20:53 + +from django.db import migrations + + +class Migration(migrations.Migration): + dependencies = [('stock', '0129_migrate_stock_tracking_to_audit_log')] + + operations = [migrations.DeleteModel(name='StockItemTracking')] diff --git a/src/backend/InvenTree/stock/models.py b/src/backend/InvenTree/stock/models.py index 09ad9fb381..5ba0db69d1 100644 --- a/src/backend/InvenTree/stock/models.py +++ b/src/backend/InvenTree/stock/models.py @@ -2,10 +2,10 @@ from __future__ import annotations -import contextvars import copy import os -from contextlib import contextmanager +from collections.abc import Callable +from contextlib import ExitStack, contextmanager, nullcontext from datetime import timedelta from decimal import Decimal, InvalidOperation @@ -36,6 +36,7 @@ import InvenTree.ready import InvenTree.tasks import order.models import report.mixins +from audit.models import NOTES_MAX_LENGTH as AUDIT_NOTES_MAX_LENGTH from common.icons import validate_icon from common.settings import get_global_setting from company import models as CompanyModels @@ -425,6 +426,7 @@ STOCK_SORT_DEFAULT = StockSortOrder.DATE_OLDEST class StockItem( + InvenTree.models.InvenTreeAuditMixin, InvenTree.models.InvenTreeAttachmentMixin, InvenTree.models.InvenTreeBarcodeMixin, InvenTree.models.InvenTreeNoteMixin, @@ -465,6 +467,27 @@ class StockItem( IMPORT_ID_FIELDS = ['serial'] STATUS_CLASS = StockStatus + # Maximum length of the notes recorded against a stock tracking event + TRACKING_NOTES_MAX_LENGTH = AUDIT_NOTES_MAX_LENGTH + + # Stock operations are recorded as named events in the audit log (see tracking_event) + AUDIT_EVENT_TYPES = StockHistoryCode + AUDIT_PARENT_FIELD = 'part' + + # Fields which are derived from other fields + AUDIT_EXCLUDE_FIELDS = ('barcode_hash', 'serial_int') + + # Tracking deltas which are not recorded in the audit log + # (status / location changes are recorded as field changes, and the customer name via the customer reference) + AUDIT_IGNORE_DELTAS = ( + 'status', + 'status_logical', + 'old_status', + 'old_status_logical', + 'old_location', + 'customer_name', + ) + class Meta: """Model meta options.""" @@ -498,6 +521,7 @@ class StockItem( Performs a number of checks: - Unique serial number requirement - Adds a transaction note when the item is first created. + - Records the user / notes (if provided) against the audit entry for an update. """ self.validate_unique() self.clean() @@ -513,60 +537,31 @@ class StockItem( notes = kwargs.pop('notes', '') + # Tracking event recorded against this save (if any) + event = None + if self.pk: - # StockItem has already been saved + # Changes to an existing item are recorded as a regular update, + # including the user and notes (if provided) - unless an event is already pending + from audit.context import has_pending_event + from audit.helpers import audit_event - # Check if "interesting" fields have been changed - # (we wish to record these as historical records) + if (user or notes) and not has_pending_event(self): + event = audit_event(self, None, notes=notes, actor=user) - try: - old = StockItem.objects.get(pk=self.pk) - old_custom_status = old.get_custom_status() - custom_status = self.get_custom_status() - - deltas = {} - - # Status changed? - if old.status != self.status: - # Custom status changed? - # Matches custom status tracking behavior of StockChangeStatusSerializer - if old_custom_status != custom_status: - deltas['status'] = custom_status - deltas['status_logical'] = self.status - else: - deltas['status'] = self.status - deltas['status_logical'] = self.status - - if old_custom_status: - deltas['old_status'] = old_custom_status - deltas['old_status_logical'] = old.status - else: - deltas['old_status'] = old.status - deltas['old_status_logical'] = old.status - - if add_note and len(deltas) > 0: - self.add_tracking_entry( - StockHistoryCode.EDITED, user, deltas=deltas, notes=notes - ) - - except (ValueError, StockItem.DoesNotExist): - pass - - super().save(*args, **kwargs) - - # If user information is provided, and no existing note exists, create one! - if add_note and self.tracking_info.count() == 0: - tracking_info = {'status': self.status} - - self.add_tracking_entry( + elif add_note: + # A new StockItem is being created + event = self.tracking_event( StockHistoryCode.CREATED, user, - deltas=tracking_info, + deltas={'status': self.status}, notes=notes, - location=self.location, quantity=float(self.quantity), ) + with event or nullcontext(): + super().save(*args, **kwargs) + def delete(self, ignore_serial_check: bool = False, **kwargs): """Custom delete method for StockItem model. @@ -586,8 +581,14 @@ class StockItem( # Re-link any child items to the parent of this item, # so the genealogy chain survives deletion of an intermediate item + from audit.helpers import audited_update + parent = StockItem.objects.filter(pk=self.parent_id).first() - StockItem.objects.filter(parent=self).update(parent=parent) + children = StockItem.objects.filter(parent=self).select_related( + 'part', 'location', 'purchase_order__supplier' + ) + + audited_update(children, parent=parent) super().delete(**kwargs) @@ -735,11 +736,15 @@ class StockItem( @classmethod @transaction.atomic - def _create_serial_numbers(cls, serials: list, **kwargs) -> QuerySet: + def _create_serial_numbers( + cls, serials: list, tracking: Callable | None = None, **kwargs + ) -> QuerySet: """Create multiple stock items with the provided serial numbers. Arguments: serials: List of serial numbers to create + tracking: Optional callable which returns the tracking event for each created item + (see StockItem._record_created_items) **kwargs: Additional keyword arguments to pass to the StockItem creation function Returns: @@ -755,7 +760,7 @@ class StockItem( serialized against each other (see _lock_serial_numbers()), so unlike other validation checks, this method's duplicate-serial protection is safe even when the caller's own pre-check raced against another - request. It does not generate any "stock tracking entries". + request. Stock tracking events are only recorded if 'tracking' is provided. Note: This is an 'internal' function and should not be used by external code / plugins. """ @@ -835,6 +840,17 @@ class StockItem( except IntegrityError as exc: raise ValidationError({'serial_numbers': str(exc)}) from exc + # Every new item shares the same related objects - avoid redundant per-item lookups + # (bulk_create_and_fetch() does not preserve the cached related objects) + from audit.helpers import copy_related_cache + + template = StockItem(**data) + + for item in items: + copy_related_cache(template, item) + + StockItem._record_created_items(items, tracking=tracking) + # Trigger a 'created' event for the new items # Note that instead of a single event for each item, # we trigger a single event for all items created @@ -844,6 +860,30 @@ class StockItem( # Return the newly created StockItem objects return items + @staticmethod + def _record_created_items(items, tracking: Callable | None = None) -> None: + """Record audit (and tracking) entries for stock items which have been created in bulk. + + Items created in bulk (e.g. via bulk_create_and_fetch) bypass model signals, + so the audit entries must be recorded explicitly. + + Arguments: + items: The newly created StockItem instances + tracking: Optional callable which returns the tracking_event() kwargs for each item + (or None, if no tracking event is to be recorded for that item) + """ + from audit.helpers import record_created + + items = list(items) + + with ExitStack() as stack: + if tracking is not None: + for item in items: + if event := tracking(item): + stack.enter_context(item.tracking_event(**event)) + + record_created(StockItem, items) + @staticmethod def convert_serial_to_int(serial: str) -> int | None: """Convert the provided serial number to an integer value. @@ -1371,14 +1411,14 @@ class StockItem( return self.part = variant - self.save() - self.add_tracking_entry( + with self.tracking_event( StockHistoryCode.CONVERTED_TO_VARIANT, user, - deltas={'part': variant.pk}, + deltas={'part': variant}, notes=_('Converted to part') + ': ' + variant.full_name, - ) + ): + self.save() def get_item_owner(self): """Return the closest "owner" for this StockItem. @@ -1509,26 +1549,25 @@ class StockItem( else: item = self.splitStock(quantity, None, user) - # Update StockItem fields with new information - item.sales_order = order - item.customer = customer - item.location = None - - item.save(add_note=False) - code = StockHistoryCode.SENT_TO_CUSTOMER deltas = {'quantity': float(quantity)} if customer is not None: - deltas['customer'] = customer.pk + deltas['customer'] = customer deltas['customer_name'] = customer.name # If an order is provided, we are shipping against a SalesOrder, not manually! if order: code = StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER - deltas['salesorder'] = order.pk + deltas['salesorder'] = order - item.add_tracking_entry(code, user, deltas, notes=notes) + # Update StockItem fields with new information + item.sales_order = order + item.customer = customer + item.location = None + + with item.tracking_event(code, user, deltas, notes=notes): + item.save(add_note=False) trigger_event( StockEvents.ITEM_ASSIGNED_TO_CUSTOMER, @@ -1605,14 +1644,14 @@ class StockItem( } if location: - tracking_info['location'] = location.pk + tracking_info['location'] = location if item.customer: - tracking_info['customer'] = item.customer.id + tracking_info['customer'] = item.customer tracking_info['customer_name'] = item.customer.name if item.consumed_by: - tracking_info['build_order'] = item.consumed_by.id + tracking_info['build_order'] = item.consumed_by # Clear out allocation information for the stock item item.consumed_by = None @@ -1635,14 +1674,13 @@ class StockItem( ) tracking_info['old_status_logical'] = old_status_logical - item.save() + with item.tracking_event( + tracking_code, user, notes=notes, deltas=tracking_info + ): + item.save() item.clearAllocations() - item.add_tracking_entry( - tracking_code, user, notes=notes, deltas=tracking_info, location=location - ) - trigger_event(StockEvents.ITEM_RETURNED_TO_STOCK, id=item.id) # Attempt to merge returned item into parent item: @@ -1923,25 +1961,28 @@ class StockItem( stock_item.consumed_by = build stock_item.location = None - stock_item.save(add_note=False) - deltas = {'stockitem': self.pk, 'quantity': float(quantity)} + deltas = {'stockitem': self, 'quantity': float(quantity)} if build is not None: - deltas['buildorder'] = build.pk + deltas['buildorder'] = build - # Add a transaction note to the other item - stock_item.add_tracking_entry( - StockHistoryCode.INSTALLED_INTO_ASSEMBLY, user, notes=notes, deltas=deltas - ) - - # Add a transaction note to this item (the assembly) - self.add_tracking_entry( - StockHistoryCode.INSTALLED_CHILD_ITEM, - user, - notes=notes, - deltas={'stockitem': stock_item.pk, 'quantity': float(quantity)}, - ) + # Record the operation against both the installed item and this item (the assembly) + with ( + stock_item.tracking_event( + StockHistoryCode.INSTALLED_INTO_ASSEMBLY, + user, + notes=notes, + deltas=deltas, + ), + self.tracking_event( + StockHistoryCode.INSTALLED_CHILD_ITEM, + user, + notes=notes, + deltas={'stockitem': stock_item, 'quantity': float(quantity)}, + ), + ): + stock_item.save(add_note=False) trigger_event( StockEvents.ITEM_INSTALLED_INTO_ASSEMBLY, @@ -1967,33 +2008,29 @@ class StockItem( 'location': _('Cannot assign stock to structural location') }) - # Add a transaction note to the parent item - self.belongs_to.add_tracking_entry( - StockHistoryCode.REMOVED_CHILD_ITEM, - user, - deltas={'stockitem': self.pk, 'quantity': float(self.quantity)}, - notes=notes, - ) + tracking_info = {'stockitem': self.belongs_to, 'quantity': float(self.quantity)} - tracking_info = { - 'stockitem': self.belongs_to.pk, - 'quantity': float(self.quantity), - } + # Record the operation against both the parent item and this item + with ( + self.belongs_to.tracking_event( + StockHistoryCode.REMOVED_CHILD_ITEM, + user, + deltas={'stockitem': self, 'quantity': float(self.quantity)}, + notes=notes, + ), + self.tracking_event( + StockHistoryCode.REMOVED_FROM_ASSEMBLY, + user, + notes=notes, + deltas=tracking_info, + ), + ): + # Mark this stock item as *not* belonging to anyone + self.belongs_to = None + self.consumed_by = None + self.location = location - self.add_tracking_entry( - StockHistoryCode.REMOVED_FROM_ASSEMBLY, - user, - notes=notes, - deltas=tracking_info, - location=location, - ) - - # Mark this stock item as *not* belonging to anyone - self.belongs_to = None - self.consumed_by = None - self.location = location - - self.save() + self.save() def allocate_disassembly_pricing(self, quantity, lines: list) -> list: """Allocate the purchase price of this stock item across disassembly lines. @@ -2212,25 +2249,23 @@ class StockItem( if status := line.get('status'): new_item.set_status(status, custom_values=custom_status_values) - # Ensure the tree structure is observed - new_item.save(add_note=False) - - deltas = {'stockitem': self.pk, 'quantity': float(line_quantity)} + deltas = {'stockitem': self, 'quantity': float(line_quantity)} # Record traceability links against the disassembled assembly if self.build: - deltas['buildorder'] = self.build.pk + deltas['buildorder'] = self.build if self.purchase_order: - deltas['purchaseorder'] = self.purchase_order.pk + deltas['purchaseorder'] = self.purchase_order - new_item.add_tracking_entry( + # Ensure the tree structure is observed + with new_item.tracking_event( StockHistoryCode.CREATED_FROM_DISASSEMBLY, user, notes=notes, deltas=deltas, - location=destination, - ) + ): + new_item.save(add_note=False) items.append(new_item) @@ -2255,11 +2290,10 @@ class StockItem( self.quantity = self.quantity - quantity - self.save(add_note=False) - - self.add_tracking_entry( + with self.tracking_event( StockHistoryCode.DISASSEMBLED, user, notes=notes, deltas=deltas - ) + ): + self.save(add_note=False) trigger_event(StockEvents.ITEM_DISASSEMBLED, id=self.pk) @@ -2326,78 +2360,126 @@ class StockItem( return self.sales_order is None - @property - def tracking_info_count(self): - """How many tracking entries are available?""" - return self.tracking_info.count() + def _tracking_deltas(self, deltas: dict | None, kwargs: dict) -> dict: + """Return the deltas for a tracking event, including any information provided via kwargs. - @property - def has_tracking_info(self): - """Is tracking info available?""" - return self.tracking_info_count > 0 + Model references (see model_reference_fields) may be provided as kwargs, + and are retained as model instances (recorded as references in the audit log). + """ + deltas = dict(deltas or {}) + + for field in StockItem.model_reference_fields(): + if (instance := kwargs.pop(field, None)) is not None: + deltas.setdefault(field, instance) + + # Quantity specified? + if quantity := kwargs.get('quantity'): + deltas['quantity'] = float(quantity) + + return deltas + + def _audit_deltas(self, deltas: dict) -> dict: + """Return the deltas to record in the audit log for a tracking event.""" + return { + key: value + for key, value in deltas.items() + if key not in self.AUDIT_IGNORE_DELTAS + } + + @staticmethod + def _is_empty_tracking_event(entry_type: int, deltas: dict, notes: str) -> bool: + """Return True if the tracking event contains no information, and should not be recorded.""" + return ( + entry_type == StockHistoryCode.STOCK_UPDATE + and len(deltas) == 0 + and not notes + ) + + @contextmanager + def tracking_event( + self, + entry_type: int, + user: User | None = None, + deltas: dict | None = None, + notes: str = '', + **kwargs, + ): + """Record a stock tracking event against this StockItem. + + - Changes to this item which are saved within the context are recorded + in the audit log as part of this event (as a single entry) + - If this item is not saved (or not changed) within the context, + an entry for the event is recorded when the context exits + - If an event is already pending for this item (e.g. save() is called within a + stock operation), the outer event takes precedence in the audit log + + Arguments: + entry_type: Code describing the "type" of historical action (see StockHistoryCode) + user: The user performing this action + deltas: Additional information about the event (model references may be provided as instances) + notes: Notes associated with the event + **kwargs: Model references (see model_reference_fields) and quantity + """ + from audit.context import has_pending_event + from audit.helpers import audit_event + + deltas = self._tracking_deltas(deltas, kwargs) + + if self._is_empty_tracking_event(entry_type, deltas, notes): + yield + return + + if has_pending_event(self): + event = nullcontext() + else: + event = audit_event( + self, + entry_type, + notes=notes, + deltas=self._audit_deltas(deltas), + actor=user, + ) + + with event: + yield def add_tracking_entry( self, entry_type: int, - user: User, + user: User | None, deltas: dict | None = None, notes: str = '', - commit: bool = True, **kwargs, - ): - """Add a history tracking entry for this StockItem. + ) -> None: + """Record a stock tracking event for this StockItem (in the audit log). + + Note: This records the event immediately, and must not be called after saving + changes to this item as part of the same operation (which would record + two separate audit entries) - use tracking_event() around the save instead. Args: entry_type (int): Code describing the "type" of historical action (see StockHistoryCode) user (User): The user performing this action - deltas (dict, optional): A map of the changes made to the model. Defaults to None. - notes (str, optional): URL associated with this tracking entry. Defaults to ''. - commit (bool, optional): If True, save the entry to the database. Defaults to True. - - Returns: - StockItemTracking: The created tracking entry + deltas (dict, optional): Additional information about the event. Defaults to None. + notes (str, optional): Notes associated with the event. Defaults to ''. + **kwargs: Model references (see model_reference_fields) and quantity """ - if deltas is None: - deltas = {} + from audit.context import has_pending_event + from audit.helpers import log_event - # Prevent empty entry - if ( - entry_type == StockHistoryCode.STOCK_UPDATE - and len(deltas) == 0 - and not notes - ): - return None + deltas = self._tracking_deltas(deltas, kwargs) - # Extract any additional information from the kwargs - self._apply_model_reference_fields(kwargs, deltas) + if self._is_empty_tracking_event(entry_type, deltas, notes): + return - # Quantity specified? - quantity = kwargs.get('quantity') - - if quantity: - deltas['quantity'] = float(quantity) - - # If this item has already been deleted (e.g. depleted via delete_on_deplete), - # self.pk is None - Django refuses to save a FK pointing at an unsaved instance, - # even for a nullable/SET_NULL field, so pass None explicitly instead of self. - entry = StockItemTracking( - item=self if self.pk else None, - part=self.part, - tracking_type=entry_type.value, - user=user, - date=InvenTree.helpers.current_time(), - notes=notes, - deltas=deltas, - ) - - if commit: - if (batch := _tracking_batch.get()) is not None: - # A batch_tracking_entries() context is active - queue rather than save now - batch.add(entry) - else: - entry.save() - - return entry + if not has_pending_event(self): + log_event( + self, + entry_type, + notes=notes, + deltas=self._audit_deltas(deltas), + actor=user, + ) @transaction.atomic def serializeStock( @@ -2485,50 +2567,26 @@ class StockItem( data['parent'] = self # Generate a new serial number for each item - items = StockItem._create_serial_numbers(serials, **data) - - # Create a new tracking entry for each item - history_items = [] - - # Every new item shares this exact part - avoid a redundant per-item - # part_id -> Part lookup (bulk_create_and_fetch() does not preserve the - # cached FK) by seeding each item's cache with the instance we already have - part = self.part + items = StockItem._create_serial_numbers( + serials, + tracking=lambda item: { + 'entry_type': StockHistoryCode.SPLIT_FROM_PARENT, + 'user': user, + 'notes': notes, + 'quantity': 1, + }, + **data, + ) # Check (once) whether there are any test results to copy across, rather # than re-querying an identical (and usually empty) result set per item has_test_results = self.test_results.exists() - for item in items: - item.part = part - - # Construct tracking entries for the new StockItem - if entry := item.add_tracking_entry( - StockHistoryCode.SPLIT_FROM_PARENT, - user, - quantity=1, - notes=notes, - location=location, - commit=False, - ): - history_items.append(entry) - - if entry := item.add_tracking_entry( - StockHistoryCode.ASSIGNED_SERIAL, - user, - notes=notes, - deltas={'serial': item.serial}, - location=location, - commit=False, - ): - history_items.append(entry) - - # Copy any test results from this item to the new one - if has_test_results: + # Copy any test results from this item to the new items + if has_test_results: + for item in items: item.copyTestResultsFrom(self) - StockItemTracking.objects.bulk_create(history_items, batch_size=250) - # Remove the equivalent number of items self.take_stock( quantity, user, code=StockHistoryCode.STOCK_SERIALIZED, notes=notes @@ -2536,14 +2594,6 @@ class StockItem( return items - @transaction.atomic - def copyHistoryFrom(self, other): - """Copy stock history from another StockItem.""" - for item in other.tracking_info.all(): - item.item = self - item.pk = None - item.save() - @transaction.atomic def copyTestResultsFrom(self, other: StockItem, filters: dict | None = None): """Copy all test results from another StockItem.""" @@ -2764,6 +2814,12 @@ class StockItem( location = kwargs.get('location', self.location) notes = kwargs.get('notes') or '' + # Any references to other items (e.g. the merged item) must be recorded + # *before* the merged items are deleted below + from audit.helpers import serialize_deltas + + transfer_deltas = serialize_deltas(kwargs.pop('transfer_deltas', None) or {}) + parent_id = self.parent.pk if self.parent else None # Keep track of pricing data for the merged data @@ -2811,8 +2867,6 @@ class StockItem( other.delete() - transfer_deltas = kwargs.pop('transfer_deltas', None) - tracking_deltas = { 'quantity': float(self.quantity), 'added': float(merged_quantity), @@ -2823,14 +2877,6 @@ class StockItem( if transfer_deltas: tracking_deltas.update(transfer_deltas) - self.add_tracking_entry( - StockHistoryCode.MERGED_STOCK_ITEMS, - user, - quantity=self.quantity, - notes=notes, - deltas=tracking_deltas, - ) - # Update the location of the item self.location = location @@ -2856,7 +2902,14 @@ class StockItem( if quantity > 0: self.purchase_price = total_price / quantity - self.save() + with self.tracking_event( + StockHistoryCode.MERGED_STOCK_ITEMS, + user, + quantity=self.quantity, + notes=notes, + deltas=tracking_deltas, + ): + self.save() @transaction.atomic def splitStock(self, quantity, location=None, user=None, **kwargs): @@ -2936,7 +2989,7 @@ class StockItem( new_stock.parent = self # Create 'deltas' dict to record changes for tracking - deltas = {'stockitem': self.pk} + deltas = {'stockitem': self} new_stock._apply_model_reference_fields(kwargs, deltas) @@ -2946,25 +2999,27 @@ class StockItem( # Optional fields which can be supplied in a 'move' call new_stock._apply_optional_transfer_fields(kwargs, deltas) - new_stock.save(add_note=False) + if record_tracking: + # Record a stock tracking event for the newly created item + event = new_stock.tracking_event( + StockHistoryCode.SPLIT_FROM_PARENT, + user, + quantity=quantity, + notes=kwargs.get('notes', ''), + deltas=deltas, + ) + else: + event = nullcontext() + + with event: + new_stock.save(add_note=False) if split_transfer_deltas is not None: split_transfer_deltas.clear() split_transfer_deltas.update(deltas) if location: - split_transfer_deltas['location'] = location.pk - - if record_tracking: - # Add a stock tracking entry for the newly created item - new_stock.add_tracking_entry( - StockHistoryCode.SPLIT_FROM_PARENT, - user, - quantity=quantity, - notes=kwargs.get('notes', ''), - location=location, - deltas=deltas, - ) + split_transfer_deltas['location'] = location # Copy the test results of this part to the new one if kwargs.get('copy_test_results', True): @@ -3015,15 +3070,14 @@ class StockItem( ] def _apply_model_reference_fields(self, kwargs: dict, deltas: dict) -> None: - """Pop any model reference kwargs (see model_reference_fields) and record their pk in deltas.""" + """Pop any model reference kwargs (see model_reference_fields) and record them in deltas. + + Model instances are retained - these are recorded as references in the audit log. + """ for field in StockItem.model_reference_fields(): instance = deltas.pop(field, None) or kwargs.pop(field, None) - if instance is None: - continue - elif hasattr(instance, 'pk'): - deltas[field] = instance.pk - else: + if instance is not None: deltas[field] = instance def _resolve_status_kwarg(self, kwargs: dict): @@ -3061,11 +3115,8 @@ class StockItem( if field in kwargs: setattr(self, field, kwargs[field]) - # If the provided value is a model instance, store the primary key in the deltas dict - if hasattr(kwargs[field], 'pk'): - deltas[field] = kwargs[field].pk - else: - deltas[field] = kwargs[field] + # Model instances are retained (recorded as references in the audit log) + deltas[field] = kwargs[field] @transaction.atomic def move(self, location, notes, user, **kwargs): @@ -3138,7 +3189,7 @@ class StockItem( if same_location: tracking_code = StockHistoryCode.STOCK_UPDATE else: - tracking_info['location'] = location.pk + tracking_info['location'] = location status = self._resolve_status_kwarg(kwargs) self._apply_status_change(status, tracking_info) @@ -3147,9 +3198,10 @@ class StockItem( # Optional fields which can be supplied in a 'move' call self._apply_optional_transfer_fields(kwargs, tracking_info) - self.add_tracking_entry(tracking_code, user, notes=notes, deltas=tracking_info) - - self.save(add_note=False) + with self.tracking_event( + tracking_code, user, notes=notes, deltas=tracking_info + ): + self.save(add_note=False) # Trigger event for the plugin system trigger_event( @@ -3266,8 +3318,8 @@ class StockItem( if location and location != self.location: old_location = self.location self.location = location - tracking_info['location'] = location.pk - tracking_info['old_location'] = old_location.pk if old_location else None + tracking_info['location'] = location + tracking_info['old_location'] = old_location status = self._resolve_status_kwarg(kwargs) self._apply_status_change(status, tracking_info) @@ -3295,40 +3347,37 @@ class StockItem( self.stocktake_date = InvenTree.helpers.current_date() self.stocktake_user = user - raw_update_result = None if self.serialized else self.updateQuantity(count) - quantity_updated = self.serialized or raw_update_result + # Record the resulting quantity, whether or not this item survives the stocktake + # (i.e. even if the item is deleted, when counted to zero with delete_on_deplete set) + tracking_info['quantity'] = 1 if self.serialized else float(count) - # True only if updateQuantity() actually performed the save above - # (as opposed to: serialized item, no change, or item deleted) - update_persisted = raw_update_result is True - - # Record the resulting quantity, whether or not this item survived the stocktake - # (self.quantity is updated by updateQuantity() even if the item was deleted) - tracking_info['quantity'] = 1 if self.serialized else float(self.quantity) - - # Save if the quantity or any other field was changed, and updateQuantity() - # didn't already do so above. - # Note that updateQuantity() may have *deleted* the item (depleted to zero), - # in which case there is nothing left to save. - if self.pk and (quantity_updated or fields_updated): - if not update_persisted: - self.save(add_note=False) - - trigger_event( - StockEvents.ITEM_COUNTED, - id=self.id, - quantity=1 if self.serialized else float(self.quantity), - ) - - # Always record a tracking entry, even if the item was deleted as a result - # of this stocktake (e.g. counted to zero with delete_on_deplete set) - - # StockItemTracking.item uses SET_NULL so the history is retained regardless. - self.add_tracking_entry( + # Always record a tracking event (even if nothing was changed) + with self.tracking_event( StockHistoryCode.STOCK_COUNT, user, notes=kwargs.get('notes', ''), deltas=tracking_info, - ) + ): + raw_update_result = None if self.serialized else self.updateQuantity(count) + quantity_updated = self.serialized or raw_update_result + + # True only if updateQuantity() actually performed the save above + # (as opposed to: serialized item, no change, or item deleted) + update_persisted = raw_update_result is True + + # Save if the quantity or any other field was changed, and updateQuantity() + # didn't already do so above. + # Note that updateQuantity() may have *deleted* the item (depleted to zero), + # in which case there is nothing left to save. + if self.pk and (quantity_updated or fields_updated): + if not update_persisted: + self.save(add_note=False) + + trigger_event( + StockEvents.ITEM_COUNTED, + id=self.id, + quantity=1 if self.serialized else float(self.quantity), + ) @transaction.atomic def add_stock(self, quantity, user, **kwargs): @@ -3365,30 +3414,21 @@ class StockItem( self._apply_status_change(status, tracking_info) self._apply_model_reference_fields(kwargs, tracking_info) - # Determine up-front whether any optional fields will actually change, - # so we can skip the second save() below when updateQuantity() (which - # already writes the full row, including the status/reference field - # changes applied above) has already persisted everything that matters. - optional_fields_changed = any( - field in kwargs for field in StockItem.optional_transfer_fields() - ) + # Optional fields which can be supplied in an 'add' call + # Note: Applied *before* the quantity is updated, so that all changes + # are written in the single save() performed by updateQuantity() + self._apply_optional_transfer_fields(kwargs, tracking_info) - if self.updateQuantity(self.quantity + quantity): - tracking_info['added'] = float(quantity) - tracking_info['quantity'] = float(self.quantity) + tracking_info['added'] = float(quantity) + tracking_info['quantity'] = float(self.quantity + quantity) - # Optional fields which can be supplied in a 'stocktake' call - self._apply_optional_transfer_fields(kwargs, tracking_info) - - if optional_fields_changed: - self.save(add_note=False) - - self.add_tracking_entry( - StockHistoryCode.STOCK_ADD, - user, - notes=kwargs.get('notes', ''), - deltas=tracking_info, - ) + with self.tracking_event( + StockHistoryCode.STOCK_ADD, + user, + notes=kwargs.get('notes', ''), + deltas=tracking_info, + ): + self.updateQuantity(self.quantity + quantity) return True @@ -3439,35 +3479,26 @@ class StockItem( self._apply_status_change(status, deltas) self._apply_model_reference_fields(kwargs, deltas) - # Determine up-front whether any optional fields will actually change, - # so we can skip the second save() below when updateQuantity() (which - # already writes the full row, including the status/reference field - # changes applied above) has already persisted everything that matters. - optional_fields_changed = any( - field in kwargs for field in StockItem.optional_transfer_fields() - ) + # Optional fields which can be supplied in a 'remove' call + # Note: Applied *before* the quantity is updated, so that all changes + # are written in the single save() performed by updateQuantity() + self._apply_optional_transfer_fields(kwargs, deltas) - quantity_updated = self.updateQuantity(self.quantity - quantity) - - # Record the resulting quantity, whether or not this item survived the removal - # (self.quantity is updated by updateQuantity() even if the item was deleted) + # Record the resulting quantity, whether or not this item survives the removal deltas['removed'] = float(quantity) - deltas['quantity'] = float(self.quantity) + deltas['quantity'] = float(self.quantity - quantity) - if quantity_updated: - # Optional fields which can be supplied in a 'stocktake' call - self._apply_optional_transfer_fields(kwargs, deltas) - - if optional_fields_changed: - self.save(add_note=False) - - # Always record a tracking entry, even if the item was deleted as a result - # of this removal (e.g. depleted to zero with delete_on_deplete set) - - # StockItemTracking.item uses SET_NULL so the history is retained regardless. + # Always record a tracking event, even if the item is deleted as a result + # of this removal (e.g. depleted to zero with delete_on_deplete set) if record_tracking: - self.add_tracking_entry( + event = self.tracking_event( code, user, notes=kwargs.get('notes', ''), deltas=deltas ) + else: + event = nullcontext() + + with event: + self.updateQuantity(self.quantity - quantity) return True @@ -3663,152 +3694,6 @@ def after_save_stock_item(sender, instance: StockItem, created, **kwargs): instance.part.schedule_pricing_update(create=True) -# Context-local batch of pending StockItemTracking entries (see batch_tracking_entries()) -_tracking_batch: contextvars.ContextVar = contextvars.ContextVar( - 'tracking_batch', default=None -) - - -class TrackingEntryBatch: - """Collects StockItemTracking entries created within a batch_tracking_entries() scope.""" - - def __init__(self): - """Initialize an empty batch.""" - self.entries: list[StockItemTracking] = [] - - def add(self, entry: StockItemTracking) -> None: - """Record a single (unsaved) tracking entry against this batch.""" - self.entries.append(entry) - - def flush(self) -> None: - """Bulk-create all entries collected so far, in a single database write.""" - entries, self.entries = self.entries, [] - - if entries: - StockItemTracking.objects.bulk_create(entries, batch_size=250) - - -@contextmanager -def batch_tracking_entries(): - """Batch StockItemTracking entries created within this scope into a single bulk_create(). - - StockItem.add_tracking_entry() saves its entry immediately by default (commit=True). - While this context is active, any such call - made directly, or indirectly via a nested - function call - is queued instead of saved immediately. The queued entries are flushed via - a single bulk_create() when the current transaction commits (or immediately, if no - transaction is active). If the transaction is instead rolled back, the queued entries are - discarded, rather than being written for a change that never happened. - - Calls that already pass commit=False to add_tracking_entry() are unaffected either way - - that contract (the caller takes ownership of saving/collecting the entry itself, as done - for example in PurchaseOrder.receive_line_items()) is unchanged. - - Nesting is not supported: a nested batch_tracking_entries() call reuses the outer batch, - and only the outermost call schedules a flush. - - This mirrors plugin.base.event.events.batch_events() - see that docstring for the - reasoning behind the on-commit flush and the context-local (rather than parameter-based) - design, which allows existing single-item entrypoints (e.g. StockItem.stocktake()) to be - reused unmodified by both single-item and bulk callers. - - Yields: - The current TrackingEntryBatch instance - """ - if _tracking_batch.get() is not None: - # Already inside a batch - extend it, rather than creating a nested one - yield _tracking_batch.get() - return - - batch = TrackingEntryBatch() - token = _tracking_batch.set(batch) - - try: - yield batch - finally: - _tracking_batch.reset(token) - transaction.on_commit(batch.flush) - - -class StockItemTracking(InvenTree.models.InvenTreeModel): - """Stock tracking entry - used for tracking history of a particular StockItem. - - Attributes: - item: ForeignKey reference to a particular StockItem - part: ForeignKey reference to the Part associated with this StockItem - date: Date that this tracking info was created - tracking_type: The type of tracking information - notes: Associated notes (input by user) - user: The user associated with this tracking info - deltas: The changes associated with this history item - - Notes: - If the underlying stock item is deleted, the "item" field will be set to null, but the tracking information will be retained. - The tracking data will be removed if the associated part is deleted, as the tracking information is not relevant without the part context. - """ - - NOTES_MAX_LENGTH = 512 - - class Meta: - """Meta data for the StockItemTracking class.""" - - verbose_name = _('Stock Item Tracking') - - @staticmethod - def get_api_url(): - """Return API url.""" - return reverse('api-stock-tracking-list') - - def save(self, *args, **kwargs): - """Ensure that the 'part' link is always correct.""" - if self.item: - self.part = self.item.part - - super().save(*args, **kwargs) - - def get_absolute_url(self): - """Return url for instance.""" - return InvenTree.helpers.pui_url(f'/stock/item/{self.item.id}') - - def label(self): - """Return label.""" - if self.tracking_type in StockHistoryCode.keys(): # noqa: SIM118 - return StockHistoryCode.label(self.tracking_type) - - return getattr(self, 'title', '') - - tracking_type = models.IntegerField(default=StockHistoryCode.LEGACY) - - item = models.ForeignKey( - StockItem, - on_delete=models.SET_NULL, - null=True, - blank=False, - related_name='tracking_info', - ) - - part = models.ForeignKey( - 'part.part', - on_delete=models.CASCADE, - related_name='stock_tracking_info', - null=True, - blank=True, - ) - - date = models.DateTimeField(auto_now_add=True, editable=False) - - notes = models.CharField( - blank=True, - null=True, - max_length=NOTES_MAX_LENGTH, - verbose_name=_('Notes'), - help_text=_('Entry notes'), - ) - - user = models.ForeignKey(User, on_delete=models.SET_NULL, blank=True, null=True) - - deltas = models.JSONField(null=True, blank=True) - - def rename_stock_item_test_result_attachment(instance, filename): """Rename test result.""" return os.path.join( diff --git a/src/backend/InvenTree/stock/serializers.py b/src/backend/InvenTree/stock/serializers.py index 0eabb08dd6..a55a6db887 100644 --- a/src/backend/InvenTree/stock/serializers.py +++ b/src/backend/InvenTree/stock/serializers.py @@ -28,6 +28,7 @@ import part.models as part_models import part.serializers as part_serializers import stock.filters import stock.status_codes +from audit.helpers import audit_event, write_batched from common.settings import get_global_setting from generic.states.fields import InvenTreeCustomStatusSerializerMixin from importer.registry import register_importer @@ -44,14 +45,7 @@ from InvenTree.tasks import batch_offload_tasks from plugin.base.event.events import batch_events from users.serializers import UserSerializer -from .models import ( - StockItem, - StockItemTestResult, - StockItemTracking, - StockLocation, - StockLocationType, - batch_tracking_entries, -) +from .models import StockItem, StockItemTestResult, StockLocation, StockLocationType logger = structlog.get_logger('inventree') @@ -326,7 +320,7 @@ class StockItemSerializer( - Includes serialization for the item location """ - export_exclude_fields = ['tags', 'tracking_items'] + export_exclude_fields = ['tags'] SKIP_CREATE_FIELDS = ['duplicate'] @@ -400,7 +394,6 @@ class StockItemSerializer( 'supplier_part_detail', 'tags', 'tests', - 'tracking_items', ] read_only_fields = [ 'allocated', @@ -486,13 +479,7 @@ class StockItemSerializer( 'label': _('Copy Test Results'), 'help_text': _('Copy test results from the original stock item'), 'default': False, - }, - { - 'name': 'copy_history', - 'label': _('Copy History'), - 'help_text': _('Copy stock history from the original stock item'), - 'default': False, - }, + } ], ) @@ -559,7 +546,6 @@ class StockItemSerializer( ) # Annotate the queryset with the number of tracking items - queryset = queryset.annotate(tracking_items=SubqueryCount('tracking_info')) # Add flag to indicate if the StockItem has expired queryset = queryset.annotate( @@ -689,9 +675,6 @@ class StockItemSerializer( read_only=True, allow_null=True, label=_('Child Items') ) stale = serializers.BooleanField(read_only=True, allow_null=True, label=_('Stale')) - tracking_items = serializers.IntegerField( - read_only=True, allow_null=True, label=_('Tracking Items') - ) purchase_price = InvenTree.serializers.InvenTreeMoneySerializer( label=_('Purchase Price'), @@ -773,7 +756,7 @@ class SerializeStockItemSerializer(serializers.Serializer): notes = serializers.CharField( required=False, - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, allow_blank=True, label=_('Notes'), help_text=_('Optional note field'), @@ -841,7 +824,7 @@ class SerializeStockItemSerializer(serializers.Serializer): with ( transaction.atomic(), batch_events(), - batch_tracking_entries(), + write_batched(), batch_offload_tasks(), ): return ( @@ -877,7 +860,7 @@ class InstallStockItemSerializer(serializers.Serializer): ) note = serializers.CharField( - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, label=_('Note'), help_text=_('Add transaction note (optional)'), required=False, @@ -958,7 +941,7 @@ class UninstallStockItemSerializer(serializers.Serializer): ) note = serializers.CharField( - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, label=_('Notes'), help_text=_('Add transaction note (optional)'), required=False, @@ -1137,7 +1120,7 @@ class DisassembleStockItemSerializer(serializers.Serializer): notes = serializers.CharField( required=False, - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, allow_blank=True, label=_('Notes'), help_text=_('Optional note field'), @@ -1305,7 +1288,7 @@ class StockChangeStatusSerializer(serializers.Serializer): ) note = serializers.CharField( - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, label=_('Notes'), help_text=_('Add transaction note (optional)'), required=False, @@ -1325,55 +1308,25 @@ class StockChangeStatusSerializer(serializers.Serializer): note = data.get('note', '') - transaction_notes = [] - - now = InvenTree.helpers.current_time() - - # Instead of performing database updates for each item, - # perform bulk database updates (much more efficient) - # Pre-cache the custom status values (to reduce DB hits) custom_status_codes = StockItem.STATUS_CLASS.custom_values() - for item in items: - # Ignore items which are already in the desired status + # Audit log entries are written in a single query + with write_batched(): + for item in items: + # Ignore items which are already in the desired status - # Careful check for custom status codes also - if item.compare_status(status): - custom_status = item.get_custom_status() - if status == custom_status or custom_status is None: - continue + # Careful check for custom status codes also + if item.compare_status(status): + custom_status = item.get_custom_status() + if status == custom_status or custom_status is None: + continue - deltas = {'status': status} + item.set_status(status, custom_values=custom_status_codes) - # before save, track old status logical - deltas['old_status_logical'] = item.status - - if item.get_custom_status(): - deltas['old_status'] = item.get_custom_status() - else: - deltas['old_status'] = item.status - - item.set_status(status, custom_values=custom_status_codes) - item.save(add_note=False) - - # after save, can track new status_logical - deltas['status_logical'] = item.status - - # Create a new transaction note for each item - transaction_notes.append( - StockItemTracking( - item=item, - tracking_type=stock.status_codes.StockHistoryCode.EDITED.value, - date=now, - deltas=deltas, - user=user, - notes=note, - ) - ) - - # Create tracking entries - StockItemTracking.objects.bulk_create(transaction_notes, batch_size=250) + # The status change is recorded as a regular update (with the provided note) + with audit_event(item, None, notes=note, actor=user): + item.save(add_note=False) class StockLocationTypeSerializer(InvenTree.serializers.InvenTreeModelSerializer): @@ -1538,73 +1491,6 @@ class LocationSerializer( ) -@register_importer() -class StockTrackingSerializer( - InvenTree.serializers.FilterableSerializerMixin, - DataImportExportSerializerMixin, - InvenTree.serializers.InvenTreeModelSerializer, -): - """Serializer for StockItemTracking model.""" - - class Meta: - """Metaclass options.""" - - model = StockItemTracking - fields = [ - 'pk', - 'item', - 'item_detail', - 'part', - 'part_detail', - 'date', - 'deltas', - 'label', - 'notes', - 'tracking_type', - 'user', - 'user_detail', - ] - read_only_fields = ['date', 'part', 'user', 'label', 'tracking_type'] - - label = serializers.CharField(read_only=True) - - item_detail = OptionalField( - serializer_class=StockItemSerializer, - serializer_kwargs={ - 'source': 'item', - 'many': False, - 'read_only': True, - 'allow_null': True, - }, - prefetch_fields=['item', 'item__part'], - ) - - part_detail = OptionalField( - serializer_class=part_serializers.PartBriefSerializer, - serializer_kwargs={ - 'source': 'part', - 'many': False, - 'read_only': True, - 'allow_null': True, - }, - default_include=False, - prefetch_fields=['part'], - ) - - user_detail = OptionalField( - serializer_class=UserSerializer, - serializer_kwargs={ - 'source': 'user', - 'many': False, - 'read_only': True, - 'allow_null': True, - }, - prefetch_fields=['user'], - ) - - deltas = serializers.JSONField(read_only=True) - - class StockAssignmentItemSerializer(serializers.Serializer): """Serializer for a single StockItem with in StockAssignment request. @@ -1689,7 +1575,7 @@ class StockAssignmentSerializer(serializers.Serializer): notes = serializers.CharField( required=False, - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, allow_blank=True, label=_('Notes'), help_text=_('Stock assignment notes'), @@ -1779,7 +1665,7 @@ class StockMergeSerializer(serializers.Serializer): notes = serializers.CharField( required=False, - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, allow_blank=True, label=_('Notes'), help_text=_('Stock merging notes'), @@ -1989,7 +1875,7 @@ class StockAdjustmentSerializer(serializers.Serializer): notes = serializers.CharField( required=False, - max_length=StockItemTracking.NOTES_MAX_LENGTH, + max_length=StockItem.TRACKING_NOTES_MAX_LENGTH, allow_blank=True, label=_('Notes'), help_text=_('Stock transaction notes'), @@ -2073,7 +1959,7 @@ class StockCountSerializer(StockAdjustmentSerializer): with ( transaction.atomic(), batch_events(), - batch_tracking_entries(), + write_batched(), batch_offload_tasks(), ): for item in items: @@ -2106,7 +1992,7 @@ class StockAddSerializer(StockAdjustmentSerializer): with ( transaction.atomic(), batch_events(), - batch_tracking_entries(), + write_batched(), batch_offload_tasks(), ): for item in data['items']: @@ -2140,7 +2026,7 @@ class StockRemoveSerializer(StockAdjustmentSerializer): with ( transaction.atomic(), batch_events(), - batch_tracking_entries(), + write_batched(), batch_offload_tasks(), ): for item in data['items']: @@ -2202,7 +2088,7 @@ class StockTransferSerializer(StockAdjustmentSerializer): with ( transaction.atomic(), batch_events(), - batch_tracking_entries(), + write_batched(), batch_offload_tasks(), ): for item in items: @@ -2251,10 +2137,10 @@ class StockTransferSerializer(StockAdjustmentSerializer): merge_kwargs['transfer_deltas'] = transfer_deltas target.merge_stock_items([piece], **merge_kwargs) else: - transfer_deltas = {'stockitem': stock_item.pk} + transfer_deltas = {'stockitem': stock_item} if location: - transfer_deltas['location'] = location.pk + transfer_deltas['location'] = location for field_name in StockItem.optional_transfer_fields(): if field_name in kwargs: diff --git a/src/backend/InvenTree/stock/tasks.py b/src/backend/InvenTree/stock/tasks.py deleted file mode 100644 index d56fed9630..0000000000 --- a/src/backend/InvenTree/stock/tasks.py +++ /dev/null @@ -1,36 +0,0 @@ -"""Background tasks for the stock app.""" - -from datetime import datetime, timedelta - -import structlog -from opentelemetry import trace - -from common.settings import get_global_setting -from InvenTree.tasks import ScheduledTask, scheduled_task - -tracer = trace.get_tracer(__name__) -logger = structlog.get_logger('inventree') - - -@tracer.start_as_current_span('delete_old_stock_tracking') -@scheduled_task(ScheduledTask.DAILY) -def delete_old_stock_tracking(): - """Remove old stock tracking entries before a certain date.""" - from stock.models import StockItemTracking - - if not get_global_setting('STOCK_TRACKING_DELETE_OLD_ENTRIES', False): - return - - delete_n_days = int(get_global_setting('STOCK_TRACKING_DELETE_DAYS', 365)) - - threshold = datetime.now() - timedelta(days=delete_n_days) - - old_entries = StockItemTracking.objects.filter(date__lte=threshold) - - if old_entries.exists(): - logger.info( - 'Deleting old stock tracking entries', - count=old_entries.count(), - threshold=threshold, - ) - old_entries.delete() diff --git a/src/backend/InvenTree/stock/test_api.py b/src/backend/InvenTree/stock/test_api.py index a3ae6bfd7e..c38ebce1c8 100644 --- a/src/backend/InvenTree/stock/test_api.py +++ b/src/backend/InvenTree/stock/test_api.py @@ -1,8 +1,8 @@ """Unit testing for the Stock API.""" import os -import random from datetime import datetime, timedelta +from decimal import Decimal from django.contrib.contenttypes.models import ContentType from django.core.exceptions import ValidationError @@ -23,12 +23,13 @@ from InvenTree.unit_test import ( InvenTreeAPIPerformanceTestCase, InvenTreeAPITestCase, findOffloadedEvent, + stock_events, + stock_history, ) from part.models import Part, PartTestTemplate from stock.models import ( StockItem, StockItemTestResult, - StockItemTracking, StockLocation, StockLocationType, ) @@ -1621,9 +1622,10 @@ class StockItemTest(StockAPITestCase): # Check that the item was associated with the correct user item = StockItem.objects.get(pk=response.data[0]['pk']) - self.assertEqual(item.tracking_info_count, 1) - tracking = item.tracking_info.first() - self.assertEqual(tracking.user, self.user) + self.assertEqual(stock_events(item).count(), 1) + tracking = stock_events(item).first() + self.assertEqual(tracking.event_type, StockHistoryCode.CREATED) + self.assertEqual(tracking.actor, self.user) # What if we explicitly set the location to a different value? @@ -2110,7 +2112,7 @@ class StockItemTest(StockAPITestCase): def test_note_max_length(self): """Test that overly long transaction notes are rejected with a validation error.""" - max_length = StockItemTracking.NOTES_MAX_LENGTH + max_length = StockItem.TRACKING_NOTES_MAX_LENGTH note = 'x' * (max_length + 1) for url, data in [ @@ -2139,7 +2141,7 @@ class StockItemTest(StockAPITestCase): sub_part = part.models.Part.objects.get(pk=50) sub_item = StockItem.objects.create(part=sub_part, serial='xyz-123', quantity=1) - n_entries = sub_item.tracking_info.count() + n_entries = stock_events(sub_item).count() self.assertIsNone(sub_item.belongs_to) @@ -2170,7 +2172,11 @@ class StockItemTest(StockAPITestCase): self.assertEqual(sub_item.belongs_to, item) - self.assertEqual(n_entries + 1, sub_item.tracking_info.count()) + self.assertEqual(n_entries + 1, stock_events(sub_item).count()) + self.assertEqual( + stock_events(sub_item).last().event_type, + StockHistoryCode.INSTALLED_INTO_ASSEMBLY, + ) # Try to install again - this time, should fail because the StockItem is not available! response = self.post( @@ -2215,7 +2221,7 @@ class StockItemTest(StockAPITestCase): item.customer = customer item.save() - n_entries = item.tracking_info_count + n_entries = stock_events(item).count() url = reverse('api-stock-return') @@ -2249,8 +2255,12 @@ class StockItemTest(StockAPITestCase): item.refresh_from_db() - # A new stock tracking entry should have been created - self.assertEqual(n_entries + 1, item.tracking_info_count) + # A new stock tracking event should have been recorded + self.assertEqual(n_entries + 1, stock_events(item).count()) + + entry = stock_events(item).last() + self.assertEqual(entry.event_type, StockHistoryCode.RETURNED_TO_STOCK) + self.assertEqual(entry.notes, 'Returned from this customer for testing') # The item is now in stock self.assertIsNone(item.customer) @@ -2300,6 +2310,18 @@ class StockItemTest(StockAPITestCase): stock_item.refresh_from_db() self.assertEqual(stock_item.part, variant) + def status_change(self, item): + """Return the most recent audit entry for the provided stock item.""" + from audit.models import AuditLogEntry + + return ( + AuditLogEntry.objects + .get_for_object(item) + .filter(action=AuditLogEntry.Action.UPDATE) + .order_by('pk') + .last() + ) + def test_set_status(self): """Test API endpoint for setting StockItem status.""" url = reverse('api-stock-change-status') @@ -2317,11 +2339,12 @@ class StockItemTest(StockAPITestCase): for item in items: item.refresh_from_db() self.assertEqual(item.status, StockStatus.OK.value) - self.assertEqual(item.tracking_info.count(), 1) + self.assertIsNone(self.status_change(item)) data = { 'items': [item.pk for item in items], 'status': StockStatus.DAMAGED.value, + 'note': 'Damaged in transit', } self.post(url, data, expected_code=201) @@ -2330,21 +2353,23 @@ class StockItemTest(StockAPITestCase): for item in items: item.refresh_from_db() self.assertEqual(item.status, StockStatus.DAMAGED.value) - self.assertEqual(item.tracking_info.count(), 2) - tracking = item.tracking_info.last() - self.assertEqual(tracking.deltas['old_status'], StockStatus.OK.value) + + # The status change is recorded as a regular update + entry = self.status_change(item) + self.assertIsNone(entry.event_type) + self.assertEqual(entry.actor, self.user) + self.assertEqual(entry.notes, 'Damaged in transit') self.assertEqual( - tracking.deltas['old_status_logical'], StockStatus.OK.value - ) - self.assertEqual(tracking.deltas['status'], StockStatus.DAMAGED.value) - self.assertEqual( - tracking.deltas['status_logical'], StockStatus.DAMAGED.value + entry.changes_dict['status'], + [StockStatus.OK.value, StockStatus.DAMAGED.value], ) # Same test, but with one item unchanged items[0].set_status(StockStatus.ATTENTION.value) items[0].save() + unchanged = self.status_change(items[0]) + data['status'] = StockStatus.ATTENTION.value self.post(url, data, expected_code=201) @@ -2352,10 +2377,15 @@ class StockItemTest(StockAPITestCase): for item in items: item.refresh_from_db() self.assertEqual(item.status, StockStatus.ATTENTION.value) - self.assertEqual(item.tracking_info.count(), 3) - tracking = item.tracking_info.last() - self.assertEqual(tracking.tracking_type, StockHistoryCode.EDITED.value) + # No entry is recorded for the unchanged item + self.assertEqual(self.status_change(items[0]).pk, unchanged.pk) + + for item in items[1:]: + self.assertEqual( + self.status_change(item).changes_dict['status'], + [StockStatus.DAMAGED.value, StockStatus.ATTENTION.value], + ) def test_set_custom_status(self): """Test API endpoint for setting StockItem custom status.""" @@ -2374,7 +2404,6 @@ class StockItemTest(StockAPITestCase): for item in items: item.refresh_from_db() self.assertEqual(item.status, StockStatus.OK.value) - self.assertEqual(item.tracking_info.count(), 1) # Test tracking with custom status # *from* standard *to* custom @@ -2389,35 +2418,36 @@ class StockItemTest(StockAPITestCase): item.refresh_from_db() self.assertEqual(item.status, self.inspect_custom_status.logical_key) self.assertEqual(item.get_custom_status(), self.inspect_custom_status.key) - tracking = item.tracking_info.last() - self.assertEqual(tracking.deltas['old_status'], StockStatus.OK.value) + + changes = self.status_change(item).changes_dict + self.assertEqual( - tracking.deltas['old_status_logical'], StockStatus.OK.value + changes['status'], + [StockStatus.OK.value, self.inspect_custom_status.logical_key], ) - self.assertEqual(tracking.deltas['status'], self.inspect_custom_status.key) self.assertEqual( - tracking.deltas['status_logical'], - self.inspect_custom_status.logical_key, + changes['status_custom_key'], [None, self.inspect_custom_status.key] ) # reverse case # *from* custom *to* standard data['status'] = StockStatus.OK.value self.post(url, data, expected_code=201) + for item in items: item.refresh_from_db() self.assertEqual(item.status, StockStatus.OK.value) self.assertIsNone(item.get_custom_status()) - tracking = item.tracking_info.last() + + changes = self.status_change(item).changes_dict + self.assertEqual( - tracking.deltas['old_status'], self.inspect_custom_status.key + changes['status'], + [self.inspect_custom_status.logical_key, StockStatus.OK.value], ) self.assertEqual( - tracking.deltas['old_status_logical'], - self.inspect_custom_status.logical_key, + changes['status_custom_key'], [self.inspect_custom_status.key, None] ) - self.assertEqual(tracking.deltas['status'], StockStatus.OK.value) - self.assertEqual(tracking.deltas['status_logical'], StockStatus.OK.value) def test_bulk_batch_change(self): """Test that we can bulk-change batch code for a set of stock items.""" @@ -2618,18 +2648,18 @@ class StockItemDisassembleTest(StockAPITestCase): # Check stock tracking entries have been created self.assertTrue( - self.item.tracking_info.filter( - tracking_type=StockHistoryCode.DISASSEMBLED.value - ).exists() + stock_events(self.item, event_type=StockHistoryCode.DISASSEMBLED).exists() ) for item in [item_1, item_50]: - entry = item.tracking_info.filter( - tracking_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY.value + entry = stock_events( + item, event_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY ).first() self.assertIsNotNone(entry) - self.assertEqual(entry.deltas['stockitem'], self.item.pk) + self.assertEqual( + entry.additional_data['deltas']['stockitem']['pk'], self.item.pk + ) def test_full_disassembly(self): """Test that a fully disassembled item is retained at zero quantity.""" @@ -2675,13 +2705,13 @@ class StockItemDisassembleTest(StockAPITestCase): # an output of the build) - instead it is recorded in the stock history self.assertIsNone(component.build) - entry = component.tracking_info.filter( - tracking_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY.value + entry = stock_events( + component, event_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY ).first() self.assertIsNotNone(entry) - self.assertEqual(entry.deltas['buildorder'], bo.pk) - self.assertEqual(entry.deltas['purchaseorder'], po.pk) + self.assertEqual(entry.additional_data['deltas']['buildorder']['pk'], bo.pk) + self.assertEqual(entry.additional_data['deltas']['purchaseorder']['pk'], po.pk) def test_explicit_pricing(self): """Test that automatic cost allocation is skipped if explicit pricing is provided.""" @@ -2750,23 +2780,23 @@ class StockItemDisassembleTest(StockAPITestCase): # Uninstalled items are updated, not created afresh self.assertTrue( - sub_1.tracking_info.filter( - tracking_type=StockHistoryCode.REMOVED_FROM_ASSEMBLY.value + stock_events( + sub_1, event_type=StockHistoryCode.REMOVED_FROM_ASSEMBLY ).exists() ) self.assertFalse( - sub_1.tracking_info.filter( - tracking_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY.value + stock_events( + sub_1, event_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY ).exists() ) self.assertTrue( - self.item.tracking_info.filter( - tracking_type=StockHistoryCode.REMOVED_CHILD_ITEM.value + stock_events( + self.item, event_type=StockHistoryCode.REMOVED_CHILD_ITEM ).exists() ) self.assertTrue( - new_item.tracking_info.filter( - tracking_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY.value + stock_events( + new_item, event_type=StockHistoryCode.CREATED_FROM_DISASSEMBLY ).exists() ) @@ -3017,7 +3047,7 @@ class StocktakeTest(StockAPITestCase): def test_notes_max_length(self): """Test that overly long notes are rejected with a validation error.""" - max_length = StockItemTracking.NOTES_MAX_LENGTH + max_length = StockItem.TRACKING_NOTES_MAX_LENGTH for endpoint in [ 'api-stock-count', @@ -3130,12 +3160,9 @@ class StocktakeTest(StockAPITestCase): item_a.refresh_from_db() self.assertEqual(item_a.location.pk, 1) - # Tracking entry records the location change - entry = StockItemTracking.objects.filter( - item=item_a, tracking_type=StockHistoryCode.STOCK_COUNT - ).latest('date') - self.assertEqual(entry.deltas.get('location'), 1) - self.assertEqual(entry.deltas.get('old_location'), 5) + # The stocktake event records the location change + entry = stock_events(item_a, event_type=StockHistoryCode.STOCK_COUNT).last() + self.assertEqual(entry.changes_dict['location'], [5, 1]) # --- location is updated for multiple items simultaneously --- response = self.post( @@ -3156,13 +3183,10 @@ class StocktakeTest(StockAPITestCase): self.assertEqual(item_a.location.pk, 2) self.assertEqual(item_b.location.pk, 2) - # Both items have a tracking entry with the new location + # Both items have a stocktake event with the new location for item, old_loc in [(item_a, 1), (item_b, 3)]: - entry = StockItemTracking.objects.filter( - item=item, tracking_type=StockHistoryCode.STOCK_COUNT - ).latest('date') - self.assertEqual(entry.deltas.get('location'), 2) - self.assertEqual(entry.deltas.get('old_location'), old_loc) + entry = stock_events(item, event_type=StockHistoryCode.STOCK_COUNT).last() + self.assertEqual(entry.changes_dict['location'], [old_loc, 2]) # --- location is unchanged when not provided --- response = self.post( @@ -3173,12 +3197,10 @@ class StocktakeTest(StockAPITestCase): # Location should still be 2 (unchanged from the previous count) self.assertEqual(item_a.location.pk, 2) - # Tracking entry has no location delta when location was not provided - entry = StockItemTracking.objects.filter( - item=item_a, tracking_type=StockHistoryCode.STOCK_COUNT - ).latest('date') - self.assertNotIn('location', entry.deltas) - self.assertNotIn('old_location', entry.deltas) + # No location change is recorded when location was not provided + entry = stock_events(item_a, event_type=StockHistoryCode.STOCK_COUNT).last() + self.assertNotIn('location', entry.changes_dict) + self.assertNotIn('location', (entry.additional_data or {}).get('deltas', {})) # --- structural location is rejected --- structural = StockLocation.objects.create(name='Structural', structural=True) @@ -3234,14 +3256,13 @@ class StocktakeTest(StockAPITestCase): self.assertIsNotNone(item.stocktake_date) # The stock history entry must agree with the saved state - entry = StockItemTracking.objects.filter( - item=item, tracking_type=StockHistoryCode.STOCK_COUNT - ).latest('date') - self.assertEqual(entry.deltas.get('quantity'), float(quantity)) - self.assertEqual(entry.deltas.get('location'), 1) - self.assertEqual(entry.deltas.get('old_location'), 5) - self.assertEqual(entry.deltas.get('status'), StockStatus.DAMAGED.value) - self.assertEqual(entry.deltas.get('old_status'), StockStatus.OK.value) + entry = stock_events(item, event_type=StockHistoryCode.STOCK_COUNT).last() + self.assertEqual(entry.additional_data['deltas']['quantity'], float(quantity)) + self.assertEqual(entry.changes_dict['location'], [5, 1]) + self.assertEqual( + entry.changes_dict['status'], + [StockStatus.OK.value, StockStatus.DAMAGED.value], + ) def test_bulk_count_query_benchmark(self): """Benchmark: measure the number of DB queries required to count 100 stock items at once.""" @@ -3475,9 +3496,6 @@ class StockTransferMergeTest(StockAPITestCase): def test_transfer_merge_does_not_copy_source_tracking(self): """Transfer merge keeps destination history and adds a single merge entry.""" - # Track total number of tracking entries created - N_TRACKING_ENTRIES = StockItemTracking.objects.count() - existing = StockItem.objects.create( part=self.part, location=self.dest, quantity=100 ) @@ -3490,7 +3508,7 @@ class StockTransferMergeTest(StockAPITestCase): ) incoming_pk = incoming.pk - tracking_count = existing.tracking_info.count() + tracking_count = stock_events(existing).count() self.post( self.url, @@ -3503,35 +3521,39 @@ class StockTransferMergeTest(StockAPITestCase): existing.refresh_from_db() + # The history of the incoming item is not copied to the existing item self.assertFalse( - existing.tracking_info.filter(notes='Source tracking entry').exists() + stock_history(existing, notes='Source tracking entry').exists() ) - self.assertEqual(existing.tracking_info.count(), tracking_count + 1) - merge_entry = existing.tracking_info.filter( - tracking_type=StockHistoryCode.MERGED_STOCK_ITEMS + self.assertEqual(stock_events(existing).count(), tracking_count + 1) + + merge_entry = stock_events( + existing, event_type=StockHistoryCode.MERGED_STOCK_ITEMS ).first() + self.assertIsNotNone(merge_entry) - self.assertEqual(merge_entry.deltas['added'], 50.0) - self.assertEqual(merge_entry.deltas['quantity'], 150.0) - self.assertEqual(merge_entry.deltas['stockitem'], incoming_pk) - self.assertEqual(merge_entry.deltas['location'], self.dest.pk) + self.assertEqual( + [Decimal(value) for value in merge_entry.changes_dict['quantity']], + [100, 150], + ) - self.assertEqual(StockItemTracking.objects.count(), N_TRACKING_ENTRIES + 4) + deltas = merge_entry.additional_data['deltas'] + self.assertEqual(deltas['added'], 50.0) + self.assertEqual(deltas['stockitem']['pk'], incoming_pk) + self.assertEqual(deltas['location']['pk'], self.dest.pk) - # Ensure tracking entries were bulk created in the correct order - entries = list(StockItemTracking.objects.order_by('-pk')[:4])[::-1] + # Ensure the expected events were recorded (in order) + self.assertEqual( + list(stock_events(existing).values_list('event_type', flat=True)), + [StockHistoryCode.CREATED, StockHistoryCode.MERGED_STOCK_ITEMS], + ) + self.assertEqual( + list(stock_events(incoming_pk).values_list('event_type', flat=True)), + [StockHistoryCode.CREATED, StockHistoryCode.STOCK_UPDATE], + ) - for idx, tt in enumerate([ - StockHistoryCode.CREATED, - StockHistoryCode.CREATED, - StockHistoryCode.STOCK_UPDATE, - StockHistoryCode.MERGED_STOCK_ITEMS, - ]): - self.assertEqual( - entries[idx].tracking_type, - tt, - f'Entry {idx} has unexpected tracking type {entries[idx].tracking_type}', - ) + # The incoming item was deleted (as part of the merge) + self.assertTrue(stock_history(incoming_pk, action=2).exists()) def test_transfer_merge_partial_reuses_split_transfer_deltas(self): """Partial merge reuses split transfer deltas on the merge tracking entry.""" @@ -3554,14 +3576,18 @@ class StockTransferMergeTest(StockAPITestCase): incoming.refresh_from_db() self.assertEqual(incoming.quantity, 70) - merge_entry = existing.tracking_info.filter( - tracking_type=StockHistoryCode.MERGED_STOCK_ITEMS + merge_entry = stock_events( + existing, event_type=StockHistoryCode.MERGED_STOCK_ITEMS ).first() - self.assertEqual(merge_entry.deltas['stockitem'], incoming.pk) - self.assertEqual(merge_entry.deltas['location'], self.dest.pk) + + deltas = merge_entry.additional_data['deltas'] + self.assertEqual(deltas['stockitem']['pk'], incoming.pk) + self.assertEqual(deltas['location']['pk'], self.dest.pk) + + # No split event is recorded against the source item self.assertFalse( - incoming.tracking_info.filter( - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM + stock_events( + incoming, event_type=StockHistoryCode.SPLIT_CHILD_ITEM ).exists() ) @@ -4089,127 +4115,6 @@ class StockTestResultTest(StockAPITestCase): ) -class StockTrackingTest(StockAPITestCase): - """Tests for the StockTracking API endpoints.""" - - fixtures = [ - *StockAPITestCase.fixtures, - 'build', - 'order', - 'return_order', - 'sales_order', - ] - - @classmethod - def setUpTestData(cls): - """Initialize some test data for the StockTracking tests.""" - super().setUpTestData() - - import build.models - import company.models - import order.models - import stock.models - import stock.status_codes - - entries = [] - - N_BO = build.models.Build.objects.count() - N_PO = order.models.PurchaseOrder.objects.count() - N_RO = order.models.ReturnOrder.objects.count() - N_SO = order.models.SalesOrder.objects.count() - - N_COMPANY = company.models.Company.objects.count() - N_LOCATION = stock.models.StockLocation.objects.count() - - # Generate a large quantity of tracking items - # Note that the pk values are not guaranteed to exist in the database - for item in StockItem.objects.all(): - for i in range(50): - entries.append( - StockItemTracking( - item=item, - notes='This is a test entry', - tracking_type=stock.status_codes.StockHistoryCode.LEGACY.value, - deltas={ - 'quantity': 50 - i, - 'buildorder': random.randint(0, N_BO + 1), - 'purchaseorder': random.randint(0, N_PO + 1), - 'returnorder': random.randint(0, N_RO + 1), - 'salesorder': random.randint(0, N_SO + 1), - 'customer': random.randint(0, N_COMPANY + 1), - 'location': random.randint(0, N_LOCATION + 1), - }, - ) - ) - - StockItemTracking.objects.bulk_create(entries) - - def get_url(self): - """Helper function to get stock tracking api url.""" - return reverse('api-stock-tracking-list') - - def test_count(self): - """Test list endpoint with limit = 1.""" - url = self.get_url() - - N = StockItemTracking.objects.count() - - # Test counting - response = self.get(url, {'limit': 1}) - self.assertEqual(response.data['count'], N) - - # Test with search and pagination - response = self.get(url, {'limit': 1, 'offset': 10, 'search': 'berries'}) - - self.assertEqual(response.data['count'], 0) - - def test_list(self): - """Test list endpoint.""" - url = self.get_url() - - N = StockItemTracking.objects.count() - self.assertGreater(N, 1000) - - response = self.client.get(url, max_query_count=25) - self.assertEqual(len(response.data), N) - - # Check expected delta values - keys = ['quantity', 'returnorder', 'buildorder', 'customer'] - - for item in response.data: - deltas = item['deltas'] - - for key in keys: - self.assertIn(key, deltas) - self.assertIsNotNone(deltas.get(key, None)) - - def test_output_options(self): - """Test output options.""" - self.run_output_test( - self.get_url(), - ['item_detail', 'user_detail'], - additional_params={'limit': 2}, - assert_fnc=lambda x: x.data['results'][0], - ) - - def test_status_codes_endpoint(self): - """The 'track/status/' endpoint must resolve to the status-codes view. - - Regression test: ensures the literal 'status/' path is not shadowed by the - 'track//' detail route it sits alongside in the same urlconf. - """ - response = self.get( - reverse('api-stock-tracking-status-codes'), expected_code=200 - ) - - self.assertIn('status_class', response.data) - self.assertIn('values', response.data) - self.assertIn('CREATED', response.data['values']) - self.assertEqual( - response.data['values']['CREATED']['key'], StockHistoryCode.CREATED.value - ) - - class StockAssignTest(StockAPITestCase): """Unit tests for the stock assignment API endpoint, where stock items are manually assigned to a customer.""" diff --git a/src/backend/InvenTree/stock/test_audit.py b/src/backend/InvenTree/stock/test_audit.py new file mode 100644 index 0000000000..d235cf225a --- /dev/null +++ b/src/backend/InvenTree/stock/test_audit.py @@ -0,0 +1,569 @@ +"""Audit log tests for the StockItem model (stock tracking events).""" + +from decimal import Decimal + +from django.contrib.contenttypes.models import ContentType +from django.urls import reverse + +from audit.models import AuditLogEntry +from InvenTree.unit_test import InvenTreeAPITestCase +from part.models import Part +from stock.models import StockItem, StockLocation +from stock.status_codes import StockHistoryCode, StockStatus + +ACTION = AuditLogEntry.Action + + +class StockAuditTestBase(InvenTreeAPITestCase): + """Base class for stock item audit tests.""" + + roles = ['stock.view', 'stock.add', 'stock.change', 'stock.delete', 'part.view'] + + @classmethod + def setUpTestData(cls): + """Create test data.""" + super().setUpTestData() + + cls.part = Part.objects.create( + name='Audit Widget', description='A widget', component=True + ) + cls.assembly = Part.objects.create( + name='Audit Assembly', description='An assembly', assembly=True + ) + cls.trackable = Part.objects.create( + name='Audit Trackable', description='A trackable part', trackable=True + ) + + cls.loc_a = StockLocation.objects.create(name='Location A') + cls.loc_b = StockLocation.objects.create(name='Location B') + + def setUp(self): + """Create a stock item for each test.""" + super().setUp() + + self.item = StockItem.objects.create( + part=self.part, quantity=100, location=self.loc_a + ) + + def entries(self, item): + """Return the audit entries for a stock item (oldest first).""" + return AuditLogEntry.objects.filter( + content_type=ContentType.objects.get_for_model(StockItem), + object_pk=str(item.pk), + ).order_by('pk') + + def new_entries(self, item, n): + """Return the audit entries for a stock item, recorded after the first n entries.""" + return list(self.entries(item)[n:]) + + +class StockAuditTest(StockAuditTestBase): + """Stock operations are recorded as named events in the audit log. + + Each operation records a single audit entry for each affected stock item. + """ + + def test_create(self): + """Creating a stock item records a single CREATE entry, with the CREATED event.""" + [entry] = self.entries(self.item) + + self.assertEqual(entry.action, ACTION.CREATE) + self.assertEqual(entry.event_type, StockHistoryCode.CREATED) + self.assertEqual(entry.event_label, 'Stock item created') + + # The entry is linked to the part + self.assertEqual(entry.parent_object_pk, str(self.part.pk)) + + # Derived fields are not recorded + self.assertNotIn('serial_int', entry.changes_dict) + self.assertNotIn('barcode_hash', entry.changes_dict) + + def test_create_via_api(self): + """Creating a stock item via the API records the user.""" + response = self.post( + reverse('api-stock-list'), + {'part': self.part.pk, 'quantity': 5, 'location': self.loc_a.pk}, + expected_code=201, + ) + + item = StockItem.objects.get(pk=response.data[0]['pk']) + + [entry] = self.entries(item) + + self.assertEqual(entry.event_type, StockHistoryCode.CREATED) + self.assertEqual(entry.actor, self.user) + + def test_create_serialized_via_api(self): + """Serialized items created via the API are recorded (with status).""" + response = self.post( + reverse('api-stock-list'), + { + 'part': self.trackable.pk, + 'quantity': 3, + 'serial_numbers': '1-3', + 'location': self.loc_a.pk, + 'status': StockStatus.DAMAGED.value, + }, + expected_code=201, + ) + + self.assertEqual(len(response.data), 3) + + for data in response.data: + item = StockItem.objects.get(pk=data['pk']) + self.assertEqual(item.status, StockStatus.DAMAGED.value) + + # A single entry, which records the creation (including the status) + [entry] = self.entries(item) + + self.assertEqual(entry.action, ACTION.CREATE) + self.assertEqual(entry.event_type, StockHistoryCode.CREATED) + self.assertEqual(entry.actor, self.user) + self.assertEqual(entry.changes_dict['status'][1], StockStatus.DAMAGED.value) + + def test_move(self): + """Moving the entire stock item records a single entry.""" + n = self.entries(self.item).count() + + self.item.move(self.loc_b, 'Moved', self.user) + + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.action, ACTION.UPDATE) + self.assertEqual(entry.event_type, StockHistoryCode.STOCK_MOVE) + self.assertEqual(entry.notes, 'Moved') + self.assertEqual(entry.actor, self.user) + self.assertEqual(entry.changes_dict['location'], [self.loc_a.pk, self.loc_b.pk]) + + # The location is recorded as a field change - not duplicated in the deltas + self.assertEqual(entry.additional_data['deltas'], {'quantity': 100.0}) + + def test_partial_move(self): + """Moving part of a stock item records an entry for each item.""" + n = self.entries(self.item).count() + + self.item.move(self.loc_b, 'Split', self.user, quantity=30) + + new_item = StockItem.objects.get(parent=self.item) + + # The original item + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.event_type, StockHistoryCode.SPLIT_CHILD_ITEM) + self.assertEqual(entry.changes_dict['quantity'], ['100.00000', '70.00000']) + self.assertEqual( + entry.additional_data['deltas']['stockitem'], + {'model_type': 'stock.stockitem', 'pk': new_item.pk, 'str': str(new_item)}, + ) + + # The new item + [new_entry] = self.entries(new_item) + + self.assertEqual(new_entry.action, ACTION.CREATE) + self.assertEqual(new_entry.event_type, StockHistoryCode.SPLIT_FROM_PARENT) + self.assertEqual(new_entry.changes_dict['location'][1], self.loc_b.pk) + + def test_add_remove_stock(self): + """Adding / removing stock records a single entry for each operation.""" + n = self.entries(self.item).count() + + self.item.add_stock(10, self.user, notes='Added', batch='B-123') + self.item.take_stock(5, self.user, notes='Removed') + + add, remove = self.new_entries(self.item, n) + + self.assertEqual(add.event_type, StockHistoryCode.STOCK_ADD) + self.assertEqual(add.changes_dict['quantity'], ['100.00000', '110.00000']) + self.assertEqual(add.changes_dict['batch'], ['', 'B-123']) + self.assertEqual(add.additional_data['deltas'], {'added': 10.0}) + + self.assertEqual(remove.event_type, StockHistoryCode.STOCK_REMOVE) + self.assertEqual(remove.notes, 'Removed') + self.assertEqual(remove.additional_data['deltas'], {'removed': 5.0}) + + def test_stocktake(self): + """A stocktake is always recorded - even if the quantity is unchanged.""" + n = self.entries(self.item).count() + + # Unchanged quantity + self.item.stocktake(100, self.user, notes='Counted') + + # Changed quantity + self.item.stocktake(90, self.user) + + unchanged, changed = self.new_entries(self.item, n) + + self.assertEqual(unchanged.event_type, StockHistoryCode.STOCK_COUNT) + self.assertEqual(unchanged.notes, 'Counted') + + # Nothing has changed - but the event (and counted quantity) is recorded + self.assertEqual(unchanged.changes_dict, {}) + self.assertEqual(unchanged.additional_data['deltas'], {'quantity': 100.0}) + + self.assertEqual(changed.event_type, StockHistoryCode.STOCK_COUNT) + self.assertEqual( + [Decimal(value) for value in changed.changes_dict['quantity']], + [Decimal(100), Decimal(90)], + ) + + def test_deplete(self): + """Depleting a stock item (which is then deleted) records the event against the deletion.""" + self.item.delete_on_deplete = True + self.item.save() + + pk = self.item.pk + n = self.entries(self.item).count() + + self.item.take_stock(100, self.user) + + self.assertFalse(StockItem.objects.filter(pk=pk).exists()) + + [entry] = list( + AuditLogEntry.objects.filter( + content_type=ContentType.objects.get_for_model(StockItem), + object_pk=str(pk), + ).order_by('pk')[n:] + ) + + self.assertEqual(entry.action, ACTION.DELETE) + self.assertEqual(entry.event_type, StockHistoryCode.STOCK_REMOVE) + + def test_status_change(self): + """Changing the status of stock items (via the API) records a single entry for each item.""" + n = self.entries(self.item).count() + + self.post( + reverse('api-stock-change-status'), + { + 'items': [self.item.pk], + 'status': StockStatus.DAMAGED.value, + 'note': 'Damaged', + }, + expected_code=201, + ) + + [entry] = self.new_entries(self.item, n) + + # A regular update (not a named event) - but the note is retained + self.assertEqual(entry.action, ACTION.UPDATE) + self.assertIsNone(entry.event_type) + self.assertEqual(entry.notes, 'Damaged') + self.assertEqual(entry.actor, self.user) + self.assertEqual( + entry.changes_dict['status'], + [StockStatus.OK.value, StockStatus.DAMAGED.value], + ) + + # Status information is recorded as field changes (not as deltas) + self.assertIsNone(entry.additional_data) + + def test_edit(self): + """Editing a stock item (e.g. changing the status) records a regular update entry.""" + n = self.entries(self.item).count() + + self.patch( + reverse('api-stock-detail', kwargs={'pk': self.item.pk}), + {'status': StockStatus.DAMAGED.value, 'batch': 'B-999'}, + ) + + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.action, ACTION.UPDATE) + self.assertIsNone(entry.event_type) + self.assertEqual(entry.actor, self.user) + self.assertEqual( + entry.changes_dict['status'], + [StockStatus.OK.value, StockStatus.DAMAGED.value], + ) + self.assertEqual(entry.changes_dict['batch'], ['', 'B-999']) + + def test_install(self): + """Installing an item records an entry against both items.""" + assembly = StockItem.objects.create(part=self.assembly, quantity=1) + + n_item = self.entries(self.item).count() + n_assembly = self.entries(assembly).count() + + assembly.installStockItem(self.item, 100, self.user, 'Installed') + + [item_entry] = self.new_entries(self.item, n_item) + [assembly_entry] = self.new_entries(assembly, n_assembly) + + self.assertEqual( + item_entry.event_type, StockHistoryCode.INSTALLED_INTO_ASSEMBLY + ) + self.assertEqual(item_entry.changes_dict['belongs_to'], [None, assembly.pk]) + + # Nothing has changed for the assembly - the event is still recorded + self.assertEqual( + assembly_entry.event_type, StockHistoryCode.INSTALLED_CHILD_ITEM + ) + self.assertEqual(assembly_entry.changes_dict, {}) + self.assertEqual( + assembly_entry.additional_data['deltas']['stockitem']['pk'], self.item.pk + ) + + # Uninstall + self.item.refresh_from_db() + + n_item = self.entries(self.item).count() + n_assembly = self.entries(assembly).count() + + self.item.uninstall_into_location(self.loc_b, self.user, 'Removed') + + [item_entry] = self.new_entries(self.item, n_item) + [assembly_entry] = self.new_entries(assembly, n_assembly) + + self.assertEqual(item_entry.event_type, StockHistoryCode.REMOVED_FROM_ASSEMBLY) + self.assertEqual(assembly_entry.event_type, StockHistoryCode.REMOVED_CHILD_ITEM) + + def test_serialize(self): + """Serializing stock records an entry for each new item, and the original item.""" + item = StockItem.objects.create(part=self.trackable, quantity=5) + + n = self.entries(item).count() + + item.serializeStock(3, ['A1', 'A2', 'A3'], self.user, notes='Serialized') + + [entry] = self.new_entries(item, n) + self.assertEqual(entry.event_type, StockHistoryCode.STOCK_SERIALIZED) + + for child in StockItem.objects.filter(parent=item): + [child_entry] = self.entries(child) + + self.assertEqual(child_entry.action, ACTION.CREATE) + self.assertEqual(child_entry.event_type, StockHistoryCode.SPLIT_FROM_PARENT) + self.assertEqual(child_entry.actor, self.user) + self.assertEqual(child_entry.changes_dict['serial'][1], child.serial) + + def test_merge(self): + """Merging stock items records the event against the merged item.""" + other = StockItem.objects.create( + part=self.part, quantity=50, location=self.loc_a + ) + other_pk = other.pk + + n = self.entries(self.item).count() + + self.item.merge_stock_items([other], user=self.user, location=self.loc_a) + + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.event_type, StockHistoryCode.MERGED_STOCK_ITEMS) + self.assertEqual(entry.changes_dict['quantity'], ['100.00000', '150.00000']) + + # The other item was deleted + self.assertTrue( + AuditLogEntry.objects.filter( + object_pk=str(other_pk), action=ACTION.DELETE + ).exists() + ) + + def test_part_history(self): + """Stock item entries are included in the history of the part.""" + self.item.move(self.loc_b, '', self.user) + + data = self.get( + reverse('api-audit-log-list'), + {'model_type': 'part.part', 'model_id': self.part.pk}, + ).data + + stock_entries = [e for e in data if e['model_type'] == 'stock.stockitem'] + + self.assertEqual(len(stock_entries), 2) + self.assertEqual(stock_entries[0]['event_label'], 'Location changed') + + def test_permissions(self): + """Stock history is visible to (non-staff) users with permission to view stock.""" + self.user.is_staff = False + self.user.save() + + url = reverse('api-audit-log-list') + params = {'model_type': 'stock.stockitem', 'model_id': self.item.pk} + + self.assertEqual(len(self.get(url, params).data), 1) + + self.clearRoles() + self.assignRole('part.view') + + self.assertEqual(self.get(url, params).data, []) + + +class StockEventCoverageTest(StockAuditTestBase): + """Regression tests for the remaining stock tracking event types.""" + + def latest(self, item): + """Return the most recent audit entry for a stock item.""" + return self.entries(item).last() + + def test_update_same_location(self): + """Moving stock into its current location records a STOCK_UPDATE event.""" + n = self.entries(self.item).count() + + self.item.move(self.loc_a, 'Same place', self.user, batch='B-2') + + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.event_type, StockHistoryCode.STOCK_UPDATE) + self.assertEqual(entry.notes, 'Same place') + self.assertEqual(entry.changes_dict['batch'], ['', 'B-2']) + + def test_customer(self): + """Sending stock to a customer (and returning it) records the expected events.""" + from company.models import Company + + customer = Company.objects.create(name='Audit Customer', is_customer=True) + + n = self.entries(self.item).count() + + self.item.allocateToCustomer(customer, user=self.user, notes='Sent') + + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.event_type, StockHistoryCode.SENT_TO_CUSTOMER) + self.assertEqual(entry.changes_dict['customer'], [None, customer.pk]) + self.assertEqual(entry.additional_data['deltas']['quantity'], 100.0) + + # Return the item from the customer + self.item.refresh_from_db() + n = self.entries(self.item).count() + + self.item.return_from_customer(self.loc_b, user=self.user, notes='Returned') + + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.event_type, StockHistoryCode.RETURNED_FROM_CUSTOMER) + self.assertEqual(entry.notes, 'Returned') + + # The customer is recorded as a field change (not duplicated in the deltas) + self.assertEqual(entry.changes_dict['customer'], [customer.pk, None]) + self.assertNotIn('customer', entry.additional_data['deltas']) + + def test_ship_against_order(self): + """Allocating stock against a sales order records the SHIPPED_AGAINST_SALES_ORDER event.""" + from company.models import Company + from order.models import SalesOrder + + customer = Company.objects.create(name='Order Customer', is_customer=True) + order = SalesOrder.objects.create(customer=customer, reference='SO-7777') + + n = self.entries(self.item).count() + + self.item.allocateToCustomer(customer, quantity=40, order=order, user=self.user) + + # The original item is split - the remaining stock records the split + [entry] = self.new_entries(self.item, n) + self.assertEqual(entry.event_type, StockHistoryCode.SPLIT_CHILD_ITEM) + + shipped = StockItem.objects.get(parent=self.item) + + # The split item records its creation, and the shipment + [created, shipped_entry] = self.entries(shipped) + + self.assertEqual(created.event_type, StockHistoryCode.SPLIT_FROM_PARENT) + self.assertEqual( + shipped_entry.event_type, StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER + ) + self.assertEqual( + shipped_entry.additional_data['deltas']['salesorder']['pk'], order.pk + ) + + def test_return_to_stock(self): + """Returning (consumed) stock to stock records the RETURNED_TO_STOCK event.""" + from company.models import Company + + customer = Company.objects.create(name='Return Customer', is_customer=True) + + self.item.customer = customer + self.item.location = None + self.item.save() + + n = self.entries(self.item).count() + + self.item.return_to_stock(self.loc_b, self.user, notes='Back in stock') + + [entry] = self.new_entries(self.item, n) + + self.assertEqual(entry.event_type, StockHistoryCode.RETURNED_TO_STOCK) + self.assertEqual(entry.notes, 'Back in stock') + self.assertEqual(entry.changes_dict['location'], [None, self.loc_b.pk]) + + def test_convert_to_variant(self): + """Converting a stock item to a variant records the CONVERTED_TO_VARIANT event.""" + template = Part.objects.create( + name='Template', description='A template', is_template=True + ) + variant = Part.objects.create( + name='Variant', description='A variant', variant_of=template + ) + + item = StockItem.objects.create(part=template, quantity=5) + n = self.entries(item).count() + + item.convert_to_variant(variant, self.user) + + [entry] = self.new_entries(item, n) + + self.assertEqual(entry.event_type, StockHistoryCode.CONVERTED_TO_VARIANT) + self.assertEqual(entry.changes_dict['part'], [template.pk, variant.pk]) + + def test_disassemble(self): + """Disassembling stock records DISASSEMBLED and CREATED_FROM_DISASSEMBLY events.""" + from part.models import BomItem + + bom_item = BomItem.objects.create( + part=self.assembly, sub_part=self.part, quantity=2 + ) + + assembly = StockItem.objects.create(part=self.assembly, quantity=5) + n = self.entries(assembly).count() + + components = assembly.disassemble( + 2, + [{'bom_item': bom_item, 'quantity': 4, 'location': self.loc_b}], + self.user, + notes='Broken down', + ) + + [entry] = self.new_entries(assembly, n) + + self.assertEqual(entry.event_type, StockHistoryCode.DISASSEMBLED) + self.assertEqual(entry.additional_data['deltas'], {'removed': 2.0}) + + [component] = components + [entry] = self.entries(component) + + self.assertEqual(entry.action, ACTION.CREATE) + self.assertEqual(entry.event_type, StockHistoryCode.CREATED_FROM_DISASSEMBLY) + self.assertEqual( + entry.additional_data['deltas']['stockitem']['pk'], assembly.pk + ) + + def test_transfer_merge(self): + """Transferring stock into an existing item (merge) records the event against the merged item.""" + target = StockItem.objects.create( + part=self.part, quantity=10, location=self.loc_b + ) + + n = self.entries(target).count() + + self.post( + reverse('api-stock-transfer'), + { + 'items': [{'pk': self.item.pk, 'quantity': 100, 'merge': True}], + 'location': self.loc_b.pk, + 'notes': 'Merged on transfer', + }, + expected_code=201, + ) + + target.refresh_from_db() + self.assertEqual(target.quantity, 110) + + [entry] = self.new_entries(target, n) + + self.assertEqual(entry.event_type, StockHistoryCode.MERGED_STOCK_ITEMS) + self.assertEqual(entry.notes, 'Merged on transfer') + self.assertEqual( + entry.additional_data['deltas']['stockitem']['pk'], self.item.pk + ) diff --git a/src/backend/InvenTree/stock/test_migrations.py b/src/backend/InvenTree/stock/test_migrations.py index f3284edb04..bbdd0014f6 100644 --- a/src/backend/InvenTree/stock/test_migrations.py +++ b/src/backend/InvenTree/stock/test_migrations.py @@ -272,3 +272,364 @@ class TestRemoveMpttFieldsMigration(MigratorTestCase): # The orphan item still has no parent orphan = StockItem.objects.get(pk=self.orphan_pk) self.assertIsNone(orphan.parent_id) + + +class TestStockTrackingAuditMigration(MigratorTestCase): + """Test data migration which copies StockItemTracking entries into the audit log. + + Ensures that every tracking entry is copied exactly once, with the event type, + user, date, notes, parent (part) reference and deltas transferred correctly. + """ + + migrate_from = ('stock', '0128_remove_stockitem_notes') + migrate_to = ('stock', '0129_migrate_stock_tracking_to_audit_log') + + def make_item(self, part, quantity, serial=None, location=None) -> int: + """Insert a StockItem via raw SQL (avoids a duplicate status_custom_key ORM bug).""" + from django.db import connection + + insert_sql = """ + INSERT INTO stock_stockitem + (part_id, quantity, status, delete_on_deplete, is_building, + link, serial_int, barcode_data, barcode_hash, serial, location_id) + VALUES (%s, %s, 10, false, false, '', 0, '', '', %s, %s) + """ + + params = [part.pk, quantity, serial, location.pk if location else None] + + with connection.cursor() as cursor: + if connection.features.can_return_rows_from_bulk_insert: + cursor.execute(insert_sql + 'RETURNING id', params) + return cursor.fetchone()[0] + + cursor.execute(insert_sql, params) + return cursor.lastrowid + + def prepare(self): + """Create legacy stock tracking entries to be migrated.""" + from datetime import datetime, timezone + + from django.conf import settings + + apps = self.old_state.apps + + User = apps.get_model('auth', 'user') + Part = apps.get_model('part', 'part') + Company = apps.get_model('company', 'company') + SalesOrder = apps.get_model('order', 'salesorder') + StockLocation = apps.get_model('stock', 'stocklocation') + StockItem = apps.get_model('stock', 'stockitem') + StockItemTracking = apps.get_model('stock', 'stockitemtracking') + + self.user = User.objects.create(username='tracker', email='tracker@example.com') + + self.part = Part.objects.create( + name='Tracked Widget', level=0, tree_id=0, lft=0, rght=0 + ) + + self.loc_1 = StockLocation.objects.create( + name='Shelf', pathstring='Store/Shelf', level=0, tree_id=1, lft=1, rght=2 + ) + self.loc_2 = StockLocation.objects.create( + name='Bin', pathstring='Store/Bin', level=0, tree_id=2, lft=1, rght=2 + ) + + self.customer = Company.objects.create(name='Acme Customer', is_customer=True) + self.sales_order = SalesOrder.objects.create( + reference='SO-1234', customer=self.customer, reference_int=1234 + ) + + item_pk = self.make_item(self.part, 100, location=self.loc_1) + other_pk = self.make_item(self.part, 1, serial='SN-42') + + item = StockItem.objects.get(pk=item_pk) + other = StockItem.objects.get(pk=other_pk) + + self.item_pk = item_pk + self.other_pk = other_pk + + def track(code, deltas=None, notes='', user=None, stock_item=item, day=1): + """Create a legacy tracking entry (with a known date).""" + entry = StockItemTracking.objects.create( + item=stock_item, + part=self.part, + tracking_type=code, + deltas=deltas, + notes=notes, + user=user, + ) + + # MySQL rejects timezone-aware values when USE_TZ is False + utc = timezone.utc if settings.USE_TZ else None + date = datetime(2025, 1, day, 12, 0, tzinfo=utc) + StockItemTracking.objects.filter(pk=entry.pk).update(date=date) + + return entry.pk + + self.entries = { + 'created': track( + 1, + {'status': 10, 'quantity': 100.0, 'location': self.loc_1.pk}, + notes='Initial stock', + user=self.user, + day=1, + ), + 'moved': track( + 20, + {'quantity': 100.0, 'location': self.loc_2.pk}, + user=self.user, + day=2, + ), + 'counted': track( + 10, + { + 'quantity': 90.0, + 'location': self.loc_1.pk, + 'old_location': self.loc_2.pk, + }, + user=self.user, + day=3, + ), + 'edited': track( + 5, + { + 'status': 55, + 'status_logical': 55, + 'old_status': 10, + 'old_status_logical': 10, + }, + user=self.user, + day=4, + ), + 'custom_status': track( + 5, + { + 'status': 1001, + 'status_logical': 50, + 'old_status': 55, + 'old_status_logical': 55, + }, + day=5, + ), + 'shipped': track( + 60, + { + 'quantity': 5.0, + 'salesorder': self.sales_order.pk, + 'customer': self.customer.pk, + 'customer_name': 'Acme Customer', + }, + user=self.user, + day=6, + ), + 'split': track(42, {'stockitem': other_pk, 'removed': 5.0}, day=7), + 'missing_reference': track( + 42, {'stockitem': 999999, 'removed': 1.0}, day=8 + ), + 'deleted_item': track(12, {'removed': 1.0}, stock_item=None, day=9), + 'legacy': track(0, None, notes='Legacy entry', stock_item=other, day=10), + } + + self.assertEqual(StockItemTracking.objects.count(), len(self.entries)) + + def migrated(self, name): + """Return the audit log entry migrated from the named tracking entry.""" + AuditLogEntry = self.new_state.apps.get_model('audit', 'auditlogentry') + + return AuditLogEntry.objects.get( + additional_data__tracking_id=self.entries[name] + ) + + def check_integrity(self): + """Every tracking entry is copied exactly once, with all common fields transferred.""" + apps = self.new_state.apps + + AuditLogEntry = apps.get_model('audit', 'auditlogentry') + StockItemTracking = apps.get_model('stock', 'stockitemtracking') + ContentType = apps.get_model('contenttypes', 'contenttype') + + stock_ct = ContentType.objects.get(app_label='stock', model='stockitem') + part_ct = ContentType.objects.get(app_label='part', model='part') + + # The legacy entries are retained + self.assertEqual(StockItemTracking.objects.count(), len(self.entries)) + + migrated = AuditLogEntry.objects.filter( + content_type=stock_ct, additional_data__has_key='tracking_id' + ) + + self.assertEqual(migrated.count(), len(self.entries)) + self.assertEqual( + sorted(entry.additional_data['tracking_id'] for entry in migrated), + sorted(self.entries.values()), + ) + + for tracking in StockItemTracking.objects.all(): + entry = migrated.get(additional_data__tracking_id=tracking.pk) + + self.assertEqual(entry.timestamp, tracking.date) + self.assertEqual(entry.actor_id, tracking.user_id) + self.assertEqual(entry.notes, tracking.notes or None) + + # The entry is recorded against the stock item + self.assertEqual(entry.content_type_id, stock_ct.pk) + self.assertEqual(entry.object_id, tracking.item_id) + self.assertEqual( + entry.object_pk, str(tracking.item_id) if tracking.item_id else '' + ) + + # The parent reference is the part + self.assertEqual(entry.parent_content_type_id, part_ct.pk) + self.assertEqual(entry.parent_object_pk, str(self.part.pk)) + + # The event type is transferred ('EDITED' entries are regular updates) + if tracking.tracking_type == 5: + self.assertIsNone(entry.event_type) + else: + self.assertEqual(entry.event_type, tracking.tracking_type) + + def check_created(self): + """A creation entry is recorded as a CREATE entry, with the initial values as field changes.""" + entry = self.migrated('created') + + self.assertEqual(entry.action, 0) + self.assertEqual(entry.event_type, 1) + self.assertEqual(entry.notes, 'Initial stock') + self.assertEqual(entry.actor_id, self.user.pk) + self.assertEqual(entry.actor_username, 'tracker') + self.assertEqual(entry.actor_email, 'tracker@example.com') + self.assertEqual(entry.object_repr, '100 x Tracked Widget') + + self.assertEqual( + entry.changes, + { + 'status': [None, 10], + 'quantity': [None, '100'], + 'location': [None, self.loc_1.pk], + }, + ) + + self.assertNotIn('deltas', entry.additional_data) + + def check_moved(self): + """A move records the destination location as a reference.""" + entry = self.migrated('moved') + + self.assertEqual(entry.action, 1) + self.assertEqual(entry.event_type, 20) + self.assertEqual(entry.changes, {}) + self.assertEqual( + entry.additional_data['deltas'], + { + 'quantity': 100.0, + 'location': { + 'model_type': 'stock.stocklocation', + 'pk': self.loc_2.pk, + 'str': 'Store/Bin', + }, + }, + ) + + def check_counted(self): + """A location change (with known previous location) is recorded as a field change.""" + entry = self.migrated('counted') + + self.assertEqual(entry.event_type, 10) + self.assertEqual(entry.changes, {'location': [self.loc_2.pk, self.loc_1.pk]}) + self.assertEqual(entry.additional_data['deltas'], {'quantity': 90.0}) + + def check_status_changes(self): + """Status changes are recorded as field changes (including custom status values).""" + entry = self.migrated('edited') + + self.assertEqual(entry.action, 1) + self.assertIsNone(entry.event_type) + self.assertEqual(entry.changes, {'status': [10, 55]}) + self.assertNotIn('deltas', entry.additional_data) + + entry = self.migrated('custom_status') + + self.assertEqual( + entry.changes, {'status': [55, 50], 'status_custom_key': [None, 1001]} + ) + + def check_references(self): + """References to other objects are converted to {model_type, pk, str}.""" + deltas = self.migrated('shipped').additional_data['deltas'] + + self.assertEqual( + deltas, + { + 'quantity': 5.0, + 'salesorder': { + 'model_type': 'order.salesorder', + 'pk': self.sales_order.pk, + 'str': 'SO-1234', + }, + 'customer': { + 'model_type': 'company.company', + 'pk': self.customer.pk, + 'str': 'Acme Customer', + }, + }, + ) + + deltas = self.migrated('split').additional_data['deltas'] + + self.assertEqual( + deltas['stockitem'], + { + 'model_type': 'stock.stockitem', + 'pk': self.other_pk, + 'str': 'Tracked Widget #SN-42', + }, + ) + self.assertEqual(deltas['removed'], 5.0) + + # A reference to an object which no longer exists + deltas = self.migrated('missing_reference').additional_data['deltas'] + + self.assertEqual( + deltas['stockitem'], + {'model_type': 'stock.stockitem', 'pk': 999999, 'str': None}, + ) + + def check_deleted_item(self): + """Entries for deleted stock items are retained against the part.""" + entry = self.migrated('deleted_item') + + self.assertEqual(entry.object_pk, '') + self.assertIsNone(entry.object_id) + self.assertEqual(entry.object_repr, 'Tracked Widget') + self.assertEqual(entry.parent_object_pk, str(self.part.pk)) + self.assertEqual(entry.event_type, 12) + + def check_legacy(self): + """Legacy entries (without user or deltas) are migrated.""" + entry = self.migrated('legacy') + + self.assertEqual(entry.event_type, 0) + self.assertIsNone(entry.actor_id) + self.assertIsNone(entry.actor_username) + self.assertEqual(entry.notes, 'Legacy entry') + self.assertEqual(entry.object_repr, 'Tracked Widget #SN-42') + self.assertEqual(entry.changes, {}) + self.assertEqual(entry.additional_data, {'tracking_id': self.entries['legacy']}) + + def test_migration(self): + """Run all checks against the migrated data. + + Note: A single test method is used, as the migrations are run for each test method. + """ + for check in [ + self.check_integrity, + self.check_created, + self.check_moved, + self.check_counted, + self.check_status_changes, + self.check_references, + self.check_deleted_item, + self.check_legacy, + ]: + with self.subTest(check=check.__name__): + check() diff --git a/src/backend/InvenTree/stock/tests.py b/src/backend/InvenTree/stock/tests.py index e312ae1f17..8c731a61a3 100644 --- a/src/backend/InvenTree/stock/tests.py +++ b/src/backend/InvenTree/stock/tests.py @@ -2,6 +2,7 @@ import datetime import threading +from decimal import Decimal from unittest import mock from django.contrib.auth import get_user_model @@ -15,24 +16,24 @@ from django_q.models import OrmQ from djmoney.money import Money from rest_framework.test import APIClient +from audit.helpers import write_batched +from audit.models import AuditLogEntry from build.models import Build from common.models import InvenTreeSetting from company.models import Company -from InvenTree.unit_test import AdminTestCase, InvenTreeTestCase +from InvenTree.unit_test import ( + AdminTestCase, + InvenTreeTestCase, + stock_events, + stock_history, +) from order.models import SalesOrder from part.models import Part, PartTestTemplate from plugin.base.event.events import batch_events from stock.events import StockEvents from stock.status_codes import StockHistoryCode, StockStatus -from .models import ( - StockItem, - StockItemTestResult, - StockItemTracking, - StockLocation, - StockLocationType, - batch_tracking_entries, -) +from .models import StockItem, StockItemTestResult, StockLocation, StockLocationType class StockTestBase(InvenTreeTestCase): @@ -295,21 +296,23 @@ class StockTest(StockTestBase): ) # Check that a tracking item was added - track = StockItemTracking.objects.filter(item=it).latest('id') + track = stock_events(it).last() - self.assertEqual(track.item, it) - self.assertEqual(track.tracking_type, StockHistoryCode.STOCK_MOVE) + self.assertEqual(track.event_type, StockHistoryCode.STOCK_MOVE) self.assertEqual(track.notes, 'Moved to the bathroom') def test_self_move(self): """Test moving stock to its current location.""" it = StockItem.objects.get(pk=1) - n = it.tracking_info.count() + n = stock_events(it).count() self.assertTrue(it.move(it.location, 'Moved to same place', None)) - # Ensure tracking info was not added - self.assertEqual(it.tracking_info.count(), n + 1) + # A STOCK_UPDATE event is recorded + self.assertEqual(stock_events(it).count(), n + 1) + self.assertEqual( + stock_events(it).last().event_type, StockHistoryCode.STOCK_UPDATE + ) def test_partial_move(self): """Test partial stock moving.""" @@ -361,12 +364,14 @@ class StockTest(StockTestBase): parent = StockItem.objects.get(id=1234) child = parent.splitStock(100, None, self.user) - parent_entry = parent.tracking_info.filter( - tracking_type=StockHistoryCode.SPLIT_CHILD_ITEM.value + parent_entry = stock_events( + parent, event_type=StockHistoryCode.SPLIT_CHILD_ITEM ).first() self.assertIsNotNone(parent_entry) - self.assertEqual(parent_entry.deltas.get('stockitem'), child.pk) + self.assertEqual( + parent_entry.additional_data['deltas']['stockitem']['pk'], child.pk + ) def test_delete_reparents_children(self): """Test that deleting an intermediate item re-links children to the grandparent.""" @@ -456,14 +461,16 @@ class StockTest(StockTestBase): self.assertEqual(item.quantity, 0) # History records the quantity which was *actually* removed - entry = item.tracking_info.latest('pk') - self.assertEqual(entry.deltas['removed'], 10.0) - self.assertEqual(entry.deltas['quantity'], 0.0) + entry = stock_events(item).last() + self.assertEqual(entry.additional_data['deltas']['removed'], 10.0) + self.assertEqual( + [Decimal(value) for value in entry.changes_dict['quantity']], [10, 0] + ) # Removing stock from an empty item fails, and adds no history - n = item.tracking_info.count() + n = stock_history(item).count() self.assertFalse(item.take_stock(5, self.user)) - self.assertEqual(item.tracking_info.count(), n) + self.assertEqual(stock_history(item).count(), n) # --- allocateToCustomer: allocate more than available --- customer = Company.objects.create(name='Clamp customer', is_customer=True) @@ -477,8 +484,8 @@ class StockTest(StockTestBase): self.assertEqual(allocated.customer, customer) # History records the quantity which was *actually* allocated - entry = allocated.tracking_info.latest('pk') - self.assertEqual(entry.deltas['quantity'], 10.0) + entry = stock_events(allocated).last() + self.assertEqual(entry.additional_data['deltas']['quantity'], 10.0) # A zero quantity is rejected outright item = StockItem.objects.create(part=part, quantity=10) @@ -503,15 +510,15 @@ class StockTest(StockTestBase): self.assertEqual(component.quantity, 10) # Both history entries record the quantity which was *actually* installed - entry = component.tracking_info.filter( - tracking_type=StockHistoryCode.INSTALLED_INTO_ASSEMBLY - ).latest('pk') - self.assertEqual(entry.deltas['quantity'], 10.0) + entry = stock_events( + component, event_type=StockHistoryCode.INSTALLED_INTO_ASSEMBLY + ).last() + self.assertEqual(entry.additional_data['deltas']['quantity'], 10.0) - entry = parent_item.tracking_info.filter( - tracking_type=StockHistoryCode.INSTALLED_CHILD_ITEM - ).latest('pk') - self.assertEqual(entry.deltas['quantity'], 10.0) + entry = stock_events( + parent_item, event_type=StockHistoryCode.INSTALLED_CHILD_ITEM + ).last() + self.assertEqual(entry.additional_data['deltas']['quantity'], 10.0) # A zero quantity is rejected outright other = StockItem.objects.create(part=part, quantity=5) @@ -527,8 +534,8 @@ class StockTest(StockTestBase): item.belongs_to = parent item.save() - n_entries = item.tracking_info.count() - n_parent_entries = parent.tracking_info.count() + n_entries = stock_history(item).count() + n_parent_entries = stock_history(parent).count() structural = StockLocation.objects.create( name='Structural location', structural=True @@ -541,8 +548,8 @@ class StockTest(StockTestBase): item.refresh_from_db() self.assertEqual(item.belongs_to, parent) self.assertNotEqual(item.location, structural) - self.assertEqual(item.tracking_info.count(), n_entries) - self.assertEqual(parent.tracking_info.count(), n_parent_entries) + self.assertEqual(stock_history(item).count(), n_entries) + self.assertEqual(stock_history(parent).count(), n_parent_entries) # Uninstalling into a non-structural location is still permitted item.uninstall_into_location(self.drawer2, self.user, 'Uninstalling') @@ -671,12 +678,12 @@ class StockTest(StockTestBase): self.assertEqual(it.quantity, 255) # Check that a tracking item was added - track = StockItemTracking.objects.filter(item=it).latest('id') + track = stock_events(it).last() - self.assertEqual(track.tracking_type, StockHistoryCode.STOCK_COUNT) + self.assertEqual(track.event_type, StockHistoryCode.STOCK_COUNT) self.assertIn('Counted items', track.notes) - n = it.tracking_info.count() + n = stock_history(it).count() self.assertFalse( it.stocktake( -1, @@ -686,8 +693,8 @@ class StockTest(StockTestBase): ) ) - # Ensure tracking info was not added - self.assertEqual(it.tracking_info.count(), n) + # Ensure no history was added + self.assertEqual(stock_history(it).count(), n) it.refresh_from_db() self.assertEqual(it.status, StockStatus.OK.value) @@ -780,9 +787,9 @@ class StockTest(StockTestBase): self.assertEqual(it.quantity, n + 45) # Check that a tracking item was added - track = StockItemTracking.objects.filter(item=it).latest('id') + track = stock_events(it).last() - self.assertEqual(track.tracking_type, StockHistoryCode.STOCK_ADD) + self.assertEqual(track.event_type, StockHistoryCode.STOCK_ADD) self.assertIn('Added some items', track.notes) self.assertFalse(it.add_stock(-10, None)) @@ -810,11 +817,9 @@ class StockTest(StockTestBase): self.assertFalse(ait.location) # Check that a tracking item was added - track = StockItemTracking.objects.filter(item=ait).latest('id') + track = stock_events(ait).last() - self.assertEqual( - track.tracking_type, StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER - ) + self.assertEqual(track.event_type, StockHistoryCode.SHIPPED_AGAINST_SALES_ORDER) self.assertIn('Allocated some stock', track.notes) def test_return_from_customer(self): @@ -869,9 +874,9 @@ class StockTest(StockTestBase): self.assertFalse(ait.sales_order) # Check that a tracking item was added - track = StockItemTracking.objects.filter(item=ait).latest('id') + track = stock_events(ait).last() - self.assertEqual(track.tracking_type, StockHistoryCode.RETURNED_FROM_CUSTOMER) + self.assertEqual(track.event_type, StockHistoryCode.RETURNED_FROM_CUSTOMER) self.assertIn('Stock removed from customer', track.notes) # Establish total stock for the part after remove from customer to check that we still have the correct quantity in stock @@ -889,11 +894,11 @@ class StockTest(StockTestBase): self.assertEqual(it.quantity, n - 15) # Check that a tracking item was added - track = StockItemTracking.objects.filter(item=it).latest('id') + track = stock_events(it).last() - self.assertEqual(track.tracking_type, StockHistoryCode.STOCK_REMOVE) + self.assertEqual(track.event_type, StockHistoryCode.STOCK_REMOVE) self.assertIn('Removed some items', track.notes) - self.assertTrue(it.has_tracking_info) + self.assertTrue(stock_events(it).exists()) # Test that negative quantity does nothing self.assertFalse(it.take_stock(-10, None)) @@ -1112,12 +1117,14 @@ class StockTest(StockTestBase): self.assertEqual(s1.quantity, 60) self.assertIsNone(s1.purchase_price) - merge_entry = s1.tracking_info.filter( - tracking_type=StockHistoryCode.MERGED_STOCK_ITEMS + merge_entry = stock_events( + s1, event_type=StockHistoryCode.MERGED_STOCK_ITEMS ).first() self.assertIsNotNone(merge_entry) - self.assertEqual(merge_entry.deltas['added'], 50.0) - self.assertEqual(merge_entry.deltas['quantity'], 60.0) + self.assertEqual(merge_entry.additional_data['deltas']['added'], 50.0) + self.assertEqual( + [Decimal(value) for value in merge_entry.changes_dict['quantity']], [10, 60] + ) part.stock_items.all().delete() @@ -1265,107 +1272,70 @@ class StockTest(StockTestBase): self.assertEqual(item.purchase_price_currency, 'GBP') -class TrackingEntryBatchTests(StockTestBase): - """Unit tests for the batch_tracking_entries() context manager.""" +class TrackingEventBatchTests(StockTestBase): + """Unit tests for batching stock tracking events (audit entries) via write_batched().""" - def test_entries_queued_and_flushed_on_commit(self): - """Tracking entries created inside batch_tracking_entries() are bulk-created on commit.""" + def history(self, item): + """Return the audit entries for the provided stock item.""" + return AuditLogEntry.objects.get_for_object(item).order_by('pk') + + def test_events_written_on_exit(self): + """Tracking events recorded inside write_batched() are written when the context exits.""" item = StockItem.objects.get(pk=2) - n = StockItemTracking.objects.count() + n = self.history(item).count() - with self.captureOnCommitCallbacks(execute=True): - with transaction.atomic(), batch_tracking_entries(): - for idx in range(10): - item.add_tracking_entry( - StockHistoryCode.STOCK_COUNT, - self.user, - deltas={'quantity': idx}, - notes=f'Batch entry {idx}', - ) + with transaction.atomic(), write_batched(): + for idx in range(10): + item.add_tracking_entry( + StockHistoryCode.STOCK_COUNT, + self.user, + deltas={'quantity': idx}, + notes=f'Batch entry {idx}', + ) - # Nothing should be written yet - the batch only flushes on commit - self.assertEqual(StockItemTracking.objects.count(), n) + # Nothing should be written yet + self.assertEqual(self.history(item).count(), n) - entries = StockItemTracking.objects.filter(item=item).order_by('id')[:10] - self.assertEqual(StockItemTracking.objects.count(), n + 10) + entries = list(self.history(item)[n:]) + self.assertEqual(len(entries), 10) for idx, entry in enumerate(entries): - self.assertEqual(entry.tracking_type, StockHistoryCode.STOCK_COUNT) - self.assertEqual(entry.deltas.get('quantity'), idx) + self.assertEqual(entry.event_type, StockHistoryCode.STOCK_COUNT) + self.assertEqual(entry.additional_data['deltas']['quantity'], idx) self.assertEqual(entry.notes, f'Batch entry {idx}') - def test_entries_discarded_on_rollback(self): - """Tracking entries queued in a batch are discarded if the transaction rolls back.""" + def test_events_discarded_on_rollback(self): + """Tracking events are discarded if the transaction rolls back.""" item = StockItem.objects.get(pk=2) - n = StockItemTracking.objects.count() + n = self.history(item).count() - with self.captureOnCommitCallbacks(execute=True): - try: - with transaction.atomic(), batch_tracking_entries(): - item.add_tracking_entry( - StockHistoryCode.STOCK_COUNT, self.user, deltas={'quantity': 1} - ) - raise ValueError('boom') - except ValueError: - pass + try: + with transaction.atomic(), write_batched(): + item.add_tracking_entry( + StockHistoryCode.STOCK_COUNT, self.user, deltas={'quantity': 1} + ) + raise ValueError('boom') + except ValueError: + pass - self.assertEqual(StockItemTracking.objects.count(), n) + self.assertEqual(self.history(item).count(), n) - def test_entries_outside_batch_fire_immediately(self): - """Tracking entries created outside any batch are unaffected - saved immediately.""" + def test_events_outside_batch_written_immediately(self): + """Tracking events recorded outside any batch are written immediately.""" item = StockItem.objects.get(pk=2) - n = StockItemTracking.objects.count() + n = self.history(item).count() item.add_tracking_entry( StockHistoryCode.STOCK_COUNT, self.user, deltas={'quantity': 1} ) - # No transaction commit or captureOnCommitCallbacks needed - it was never queued - self.assertEqual(StockItemTracking.objects.count(), n + 1) + self.assertEqual(self.history(item).count(), n + 1) - def test_nested_batch_share_one_flush(self): - """A nested batch_tracking_entries() call reuses the outer batch, rather than flushing twice.""" - item = StockItem.objects.get(pk=2) - - n = StockItemTracking.objects.count() - - with self.captureOnCommitCallbacks(execute=True): - with transaction.atomic(), batch_tracking_entries(): - item.add_tracking_entry( - StockHistoryCode.STOCK_COUNT, self.user, deltas={'quantity': 1} - ) - with batch_tracking_entries(): - item.add_tracking_entry( - StockHistoryCode.STOCK_COUNT, self.user, deltas={'quantity': 2} - ) - self.assertEqual(StockItemTracking.objects.count(), n) - - self.assertEqual(StockItemTracking.objects.count(), n + 2) - - def test_commit_false_is_unaffected_by_batch(self): - """commit=False callers keep manual ownership of the entry, even inside a batch.""" - item = StockItem.objects.get(pk=2) - - n = StockItemTracking.objects.count() - - with transaction.atomic(), batch_tracking_entries(): - entry = item.add_tracking_entry( - StockHistoryCode.STOCK_COUNT, - self.user, - deltas={'quantity': 1}, - commit=False, - ) - - # The entry was returned uncommitted - the batch never saw it, so it was never written - self.assertIsNotNone(entry) - self.assertIsNone(entry.pk) - self.assertEqual(StockItemTracking.objects.count(), n) - - def test_stocktake_batch_tracking_entries(self): - """Integration test: batching stocktake() tracking entries via the StockCountSerializer path.""" + def test_stocktake_batch(self): + """Integration test: batching stocktake() events via the StockCountSerializer path.""" part = Part.objects.create( name='Batch tracking part', description='For batch tracking testing' ) @@ -1375,18 +1345,14 @@ class TrackingEntryBatchTests(StockTestBase): for idx in range(10) ] - n = StockItemTracking.objects.count() + with transaction.atomic(), write_batched(): + for idx, item in enumerate(items): + item.stocktake(100 + idx, self.user, notes='Batch stocktake') - with self.captureOnCommitCallbacks(execute=True): - with transaction.atomic(), batch_tracking_entries(): - for idx, item in enumerate(items): - item.stocktake(100 + idx, self.user, notes='Batch stocktake') - - entries = StockItemTracking.objects.filter( - item__in=items, tracking_type=StockHistoryCode.STOCK_COUNT - ).order_by('id') - self.assertEqual(entries.count(), 10) - self.assertEqual(StockItemTracking.objects.count(), n + 10) + for item in items: + entry = self.history(item).last() + self.assertEqual(entry.event_type, StockHistoryCode.STOCK_COUNT) + self.assertEqual(entry.notes, 'Batch stocktake') class StockBarcodeTest(StockTestBase): @@ -2368,7 +2334,7 @@ class StockItemTakeStockConcurrencyTest(TransactionTestCase): lock is acquired) in-memory copy, could both cap to the same amount - the first to acquire the lock removes it all, and the second, upon acquiring the lock, would find nothing left, but would still record a phantom - removal in StockItemTracking and report success. + removal in the stock history and report success. take_stock() now performs the cap *after* lock_quantity() has refreshed self.quantity, so the loser of the race is correctly capped to zero and @@ -2449,8 +2415,7 @@ class StockItemTakeStockConcurrencyTest(TransactionTestCase): # recorded 'removed' amount must not exceed the quantity that # actually existed total_removed = sum( - entry.deltas.get('removed', 0) - for entry in self.item.tracking_info.all() - if entry.deltas + (entry.additional_data or {}).get('deltas', {}).get('removed', 0) + for entry in stock_events(self.item) ) self.assertEqual(total_removed, 10) diff --git a/src/backend/InvenTree/users/ruleset.py b/src/backend/InvenTree/users/ruleset.py index f1a85b4e13..08615e7727 100644 --- a/src/backend/InvenTree/users/ruleset.py +++ b/src/backend/InvenTree/users/ruleset.py @@ -148,11 +148,7 @@ def get_ruleset_models() -> dict: 'pricing_partinternalpricebreak', ], RuleSetEnum.STOCK_LOCATION: ['stock_stocklocation', 'stock_stocklocationtype'], - RuleSetEnum.STOCK: [ - 'stock_stockitem', - 'stock_stockitemtracking', - 'stock_stockitemtestresult', - ], + RuleSetEnum.STOCK: ['stock_stockitem', 'stock_stockitemtestresult'], RuleSetEnum.PURCHASE_ORDER: [ 'company_company', 'company_contact', diff --git a/src/frontend/lib/enums/ApiEndpoints.tsx b/src/frontend/lib/enums/ApiEndpoints.tsx index 4e6ab23b7b..62c83dcb5b 100644 --- a/src/frontend/lib/enums/ApiEndpoints.tsx +++ b/src/frontend/lib/enums/ApiEndpoints.tsx @@ -145,7 +145,6 @@ export enum ApiEndpoints { // Stock item API endpoints stock_item_list = 'stock/', - stock_tracking_list = 'stock/track/', stock_test_result_list = 'stock/test/', stock_transfer = 'stock/transfer/', stock_remove = 'stock/remove/', diff --git a/src/frontend/lib/enums/ModelInformation.tsx b/src/frontend/lib/enums/ModelInformation.tsx index 93f670402c..7e2aebbd60 100644 --- a/src/frontend/lib/enums/ModelInformation.tsx +++ b/src/frontend/lib/enums/ModelInformation.tsx @@ -151,7 +151,7 @@ export const ModelInformationDict: ModelDict = { stockhistory: { label: () => t`Stock History`, label_multiple: () => t`Stock Histories`, - api_endpoint: ApiEndpoints.stock_tracking_list, + api_endpoint: ApiEndpoints.audit_log_list, icon: 'history' }, build: { diff --git a/src/frontend/src/components/panels/HistoryPanel.tsx b/src/frontend/src/components/panels/HistoryPanel.tsx index b01cb643eb..9853dfa379 100644 --- a/src/frontend/src/components/panels/HistoryPanel.tsx +++ b/src/frontend/src/components/panels/HistoryPanel.tsx @@ -9,25 +9,24 @@ import { useGlobalSettingsState, useUserSettingsState } from '../../states/SettingsStates'; -import { useUserState } from '../../states/UserState'; import { AuditLogTable } from '../../tables/general/AuditLogTable'; /** * Determine whether the history panel should be displayed on detail pages: - * - Only available to staff users + * - The audit log must be enabled * - Must be enabled via the 'DISPLAY_HISTORY_TAB' user setting + * + * Note: The displayed entries are filtered by the permissions of the user */ export function useShowHistoryPanel(): boolean { - const user = useUserState(); const userSettings = useUserSettingsState(); const globalSettings = useGlobalSettingsState(); return useMemo( () => - user.isStaff() && globalSettings.isSet('AUDIT_LOG_ENABLED') && userSettings.isSet('DISPLAY_HISTORY_TAB', false), - [user, userSettings, globalSettings] + [userSettings, globalSettings] ); } diff --git a/src/frontend/src/forms/StockForms.tsx b/src/frontend/src/forms/StockForms.tsx index a8f015ba4f..4470a3a224 100644 --- a/src/frontend/src/forms/StockForms.tsx +++ b/src/frontend/src/forms/StockForms.tsx @@ -342,7 +342,6 @@ export function useStockFields({ originalId: duplicateStockItem.pk, extraFields: { copy_notes: { value: true }, - copy_history: { value: false }, copy_tests: { value: false, hidden: !duplicateStockItem?.part_detail?.testable diff --git a/src/frontend/src/pages/Index/Settings/SystemSettings.tsx b/src/frontend/src/pages/Index/Settings/SystemSettings.tsx index 10a58cbe3e..3411020a4d 100644 --- a/src/frontend/src/pages/Index/Settings/SystemSettings.tsx +++ b/src/frontend/src/pages/Index/Settings/SystemSettings.tsx @@ -297,13 +297,6 @@ export default function SystemSettings() { 'STOCKTAKE_DELETE_DAYS' ]} /> - ) { return ( - - - {t`Stock Tracking`} - - - - - {t`Stocktake Entries`} diff --git a/src/frontend/src/pages/stock/StockDetail.tsx b/src/frontend/src/pages/stock/StockDetail.tsx index 4244a62226..33c4ec598d 100644 --- a/src/frontend/src/pages/stock/StockDetail.tsx +++ b/src/frontend/src/pages/stock/StockDetail.tsx @@ -5,7 +5,6 @@ import { IconBookmark, IconBoxPadding, IconChecklist, - IconHistory, IconInfoCircle, IconPackages, IconShoppingCart, @@ -41,6 +40,9 @@ import InstanceDetail from '../../components/nav/InstanceDetail'; import NavigationTree from '../../components/nav/NavigationTree'; import { PageDetail } from '../../components/nav/PageDetail'; import AttachmentPanel from '../../components/panels/AttachmentPanel'; +import HistoryPanel, { + useShowHistoryPanel +} from '../../components/panels/HistoryPanel'; import NotesPanel from '../../components/panels/NotesPanel'; import { PanelGroup } from '../../components/panels/PanelGroup'; import LocateItemButton from '../../components/plugins/LocateItemButton'; @@ -69,7 +71,6 @@ import SalesOrderAllocationTable from '../../tables/sales/SalesOrderAllocationTa import InstalledItemsTable from '../../tables/stock/InstalledItemsTable'; import { StockItemTable } from '../../tables/stock/StockItemTable'; import StockItemTestResultTable from '../../tables/stock/StockItemTestResultTable'; -import { StockTrackingTable } from '../../tables/stock/StockTrackingTable'; import TransferOrderAllocationTable from '../../tables/stock/TransferOrderAllocationTable'; import { StockDetailsPanel } from './StockDetailsPanel'; @@ -182,6 +183,8 @@ export default function StockDetail() { return stockitem?.part_detail?.assembly; }, [trackedBomItemQuery, stockitem]); + const showHistory = useShowHistoryPanel(); + const stockPanels: PanelType[] = useMemo(() => { return [ { @@ -197,16 +200,6 @@ export default function StockDetail() { /> ) }, - { - name: 'tracking', - label: t`Stock Tracking`, - icon: , - content: stockitem.pk ? ( - - ) : ( - - ) - }, { name: 'allocations', label: t`Allocations`, @@ -320,9 +313,15 @@ export default function StockDetail() { model_type: ModelType.stockitem, model_id: stockitem.pk, note_count: instanceInfo.note_count + }), + HistoryPanel({ + model_type: ModelType.stockitem, + model_id: stockitem.pk, + visible: showHistory }) ]; }, [ + showHistory, showSalesAllocations, showBuildAllocations, showInstalledItems, diff --git a/src/frontend/src/tables/general/AuditLogTable.tsx b/src/frontend/src/tables/general/AuditLogTable.tsx index 0088b259ab..081116b1f3 100644 --- a/src/frontend/src/tables/general/AuditLogTable.tsx +++ b/src/frontend/src/tables/general/AuditLogTable.tsx @@ -233,6 +233,86 @@ function getChanges(record: any): AuditChange[] { return changes; } +/** + * Return a human readable label for an event delta key (e.g. 'sales_order' -> 'Sales order') + */ +function deltaLabel(key: string): string { + // Labels for commonly used delta keys (e.g. stock tracking events) + const labels: Record = { + added: t`Added`, + build_order: t`Build Order`, + buildorder: t`Build Order`, + customer: t`Customer`, + location: t`Location`, + part: t`Part`, + purchaseorder: t`Purchase Order`, + quantity: t`Quantity`, + removed: t`Removed`, + returnorder: t`Return Order`, + salesorder: t`Sales Order`, + stockitem: t`Stock Item`, + transferorder: t`Transfer Order` + }; + + if (key in labels) { + return labels[key]; + } + + const label = key.replaceAll('_', ' '); + return label.charAt(0).toUpperCase() + label.slice(1); +} + +/** + * Render the deltas and notes recorded against a named event (if any) + */ +function AuditEventDetails({ + record, + navigate +}: Readonly<{ + record: any; + navigate: NavigateFunction; +}>): ReactNode { + const deltas: [string, any][] = Object.entries(record.deltas ?? {}); + + if (deltas.length == 0 && !record.notes) { + return null; + } + + return ( + + {deltas.map(([key, value]) => { + // A reference to another object: {model_type, pk, str} + const isReference = + value !== null && typeof value === 'object' && 'model_type' in value; + + return ( + + + {deltaLabel(key)}: + + {isReference && !value.str ? ( + {t`Hidden`} + ) : ( + + )} + + ); + })} + {record.notes && ( + + {record.notes} + + )} + + ); +} + /** * Render a summary of the changes for an audit entry */ @@ -243,17 +323,30 @@ function AuditChangesSummary({ record: any; navigate: NavigateFunction; }>): ReactNode { + const eventDetails = ( + + ); + if (record.action == AuditAction.DELETE) { + // A named event may be recorded against the deletion (e.g. stock depleted to zero) return ( - - {t`Object was deleted`} - + + {eventDetails} + + {t`Object was deleted`} + + ); } const changes = getChanges(record); if (changes.length == 0) { + // A named event may be recorded without any field changes + if (record.event_type !== null && record.event_type !== undefined) { + return eventDetails; + } + return ( {t`No changes recorded`} @@ -263,6 +356,7 @@ function AuditChangesSummary({ return ( + {eventDetails} {changes.slice(0, MAX_INLINE_CHANGES).map((change) => ( ( - {actionLabel(record.action)} + {record.event_label ?? actionLabel(record.action)} ) }, diff --git a/src/frontend/src/tables/stock/StockTrackingTable.tsx b/src/frontend/src/tables/stock/StockTrackingTable.tsx deleted file mode 100644 index 581c075124..0000000000 --- a/src/frontend/src/tables/stock/StockTrackingTable.tsx +++ /dev/null @@ -1,317 +0,0 @@ -import { t } from '@lingui/core/macro'; -import { Table, Text } from '@mantine/core'; -import { type ReactNode, useCallback, useMemo } from 'react'; -import { useNavigate } from 'react-router-dom'; - -import { ApiEndpoints } from '@lib/enums/ApiEndpoints'; -import { ModelType } from '@lib/enums/ModelType'; -import { apiUrl } from '@lib/functions/Api'; -import { formatDecimal } from '@lib/functions/Formatting'; -import useTable from '@lib/hooks/UseTable'; -import type { TableFilter } from '@lib/types/Filters'; -import type { TableColumn } from '@lib/types/Tables'; -import { RenderBuildOrder } from '../../components/render/Build'; -import { RenderCompany } from '../../components/render/Company'; -import { - RenderPurchaseOrder, - RenderReturnOrder, - RenderSalesOrder, - RenderTransferOrder -} from '../../components/render/Order'; -import { RenderPart } from '../../components/render/Part'; -import { StatusRenderer } from '../../components/render/StatusRenderer'; -import { - RenderStockItem, - RenderStockLocation -} from '../../components/render/Stock'; -import { RenderUser } from '../../components/render/User'; -import { - DateColumn, - DescriptionColumn, - IPNColumn, - PartColumn, - StockColumn -} from '../../components/tables/ColumnRenderers'; -import { - IncludeVariantsFilter, - MaxDateFilter, - MinDateFilter, - UserFilter -} from '../../components/tables/Filter'; -import { InvenTreeTable } from '../../components/tables/InvenTreeTable'; - -type StockTrackingEntry = { - label: string; - key: string; - details: ReactNode; -}; - -export function StockTrackingTable({ - itemId, - partId -}: Readonly<{ - itemId?: number; - partId?: number; -}>) { - const navigate = useNavigate(); - const table = useTable(partId ? 'part_stock_tracking' : 'stock_tracking'); - - // Render "details" for a stock tracking record - const renderDetails = useCallback( - (record: any) => { - const deltas: any = record?.deltas ?? {}; - - const entries: StockTrackingEntry[] = [ - { - label: t`Stock Item`, - key: 'stockitem', - details: - deltas.stockitem_detail && - RenderStockItem({ instance: deltas.stockitem_detail, link: true }) - }, - { - label: t`Stock Item`, - key: 'item', - details: - deltas.item_detail && - RenderStockItem({ - instance: deltas.item_detail, - link: true - }) - }, - { - label: t`Status`, - key: 'status', - details: - deltas.status && - StatusRenderer({ - status: deltas.status, - type: ModelType.stockitem, - fallbackStatus: deltas.status_logical - }) - }, - { - label: t`Old Status`, - key: 'old_status', - details: - deltas.old_status && - StatusRenderer({ - status: deltas.old_status, - type: ModelType.stockitem, - fallbackStatus: deltas.old_status_logical - }) - }, - { - label: t`Quantity`, - key: 'quantity', - details: formatDecimal(deltas.quantity) - }, - { - label: t`Added`, - key: 'added', - details: deltas.added - }, - { - label: t`Removed`, - key: 'removed', - details: deltas.removed - }, - { - label: t`Part`, - key: 'part', - details: - deltas.part_detail && - RenderPart({ - instance: deltas.part_detail, - link: true, - navigate: navigate - }) - }, - { - label: t`Location`, - key: 'location', - details: - deltas.location_detail && - RenderStockLocation({ - instance: deltas.location_detail, - link: true, - navigate: navigate - }) - }, - { - label: t`Build Order`, - key: 'buildorder', - details: - deltas.buildorder_detail && - RenderBuildOrder({ - instance: deltas.buildorder_detail, - link: true, - navigate: navigate - }) - }, - { - label: t`Purchase Order`, - key: 'purchaseorder', - details: - deltas.purchaseorder_detail && - RenderPurchaseOrder({ - instance: deltas.purchaseorder_detail, - link: true, - navigate: navigate - }) - }, - { - label: t`Sales Order`, - key: 'salesorder', - details: - deltas.salesorder_detail && - RenderSalesOrder({ - instance: deltas.salesorder_detail, - link: true, - navigate: navigate - }) - }, - { - label: t`Return Order`, - key: 'returnorder', - details: - deltas.returnorder_detail && - RenderReturnOrder({ - instance: deltas.returnorder_detail, - link: true, - navigate: navigate - }) - }, - { - label: t`Transfer Order`, - key: 'transferorder', - details: - deltas.transferorder_detail && - RenderTransferOrder({ - instance: deltas.transferorder_detail, - link: true, - navigate: navigate - }) - }, - { - label: t`Customer`, - key: 'customer', - details: - deltas.customer_detail && - RenderCompany({ - instance: deltas.customer_detail, - link: true, - navigate: navigate - }) - } - ]; - - return ( - - - {entries.map( - (entry) => - entry.details && ( - - - {entry.label} - - {entry.details} - - ) - )} - -
- ); - }, - [navigate] - ); - - const filters: TableFilter[] = useMemo(() => { - return [ - MinDateFilter(), - MaxDateFilter(), - IncludeVariantsFilter(), - UserFilter({ - name: 'user', - label: t`User`, - description: t`Filter by user` - }) - ]; - }, []); - - const tableColumns: TableColumn[] = useMemo(() => { - return [ - DateColumn({ - switchable: false - }), - PartColumn({ - title: t`Part`, - part: 'part_detail', - switchable: true, - hidden: !partId - }), - IPNColumn({ - defaultVisible: false, - hidden: !partId - }), - StockColumn({ - title: t`Stock Item`, - accessor: 'item_detail', - nullMessage: ( - {t`Stock item no longer exists`} - ), - sortable: false, - switchable: false, - hidden: !partId - }), - DescriptionColumn({ - accessor: 'label' - }), - { - accessor: 'details', - title: t`Details`, - switchable: false, - render: renderDetails - }, - { - accessor: 'notes', - title: t`Notes`, - sortable: false, - switchable: true - }, - { - accessor: 'user', - title: t`User`, - render: (record: any) => { - if (!record.user_detail) { - return {t`No user information`}; - } - - return RenderUser({ instance: record.user_detail }); - } - } - ]; - }, []); - - return ( - - ); -} diff --git a/src/frontend/tests/pages/pui_stock.spec.ts b/src/frontend/tests/pages/pui_stock.spec.ts index 701b8d5149..517ae4da9e 100644 --- a/src/frontend/tests/pages/pui_stock.spec.ts +++ b/src/frontend/tests/pages/pui_stock.spec.ts @@ -1,5 +1,5 @@ import { expect, test } from '../baseFixtures.js'; -import { stevenuser } from '../defaults.js'; +import { adminuser, stevenuser } from '../defaults.js'; import { clearTableFilters, clickButtonIfVisible, @@ -30,7 +30,6 @@ test('Stock - Basic Tests', async ({ browser }) => { await navigate(page, 'stock/item/1194/details'); await page.getByText('D.123 | Doohickey').waitFor(); await page.getByText('Batch Code: BX-123-2024-2-7').waitFor(); - await loadTab(page, 'Stock Tracking'); await loadTab(page, 'Test Results'); await page.getByText('395c6d5586e5fb656901d047be27e1f7').waitFor(); await loadTab(page, 'Installed Items'); @@ -47,12 +46,6 @@ test('Stock - Basic Tests', async ({ browser }) => { // Automatically navigate through to the newly created stock item await page.getByText('Quantity: 987').first().waitFor(); - await loadTab(page, 'Stock Tracking'); - await page - .getByRole('cell', { name: 'Stock item created' }) - .first() - .waitFor(); - await page.getByRole('cell', { name: 'allaccess Ally Access' }).waitFor(); }); test('Stock - Location Tree', async ({ browser }) => { @@ -508,24 +501,59 @@ test('Stock - Return Items', async ({ browser }) => { await page.getByText('This field is required.').first().waitFor(); }); -test('Stock - Tracking', async ({ browser }) => { - const page = await doCachedLogin(browser, { url: 'stock/item/176/details' }); +test('Stock - History (Stock Item)', async ({ browser }) => { + // Note: The 'admin' user has the "Show Change History" setting enabled + const page = await doCachedLogin(browser, { + user: adminuser, + url: 'stock/item/193/history' + }); + + await page.getByText('widget.purple | Purple Widget').first().waitFor(); + + // Stock tracking events are displayed in the "History" tab + await page.getByText('Installed into assembly').first().waitFor(); + await page.getByText('Assigned serial number').first().waitFor(); + await page.getByText('Stock item created').first().waitFor(); + + // Event deltas link to the related stock item + await page.getByText('Widget Board (assembled) #159').first().waitFor(); +}); + +test('Stock - History (Part)', async ({ browser }) => { + // Note: The 'admin' user has the "Show Change History" setting enabled + const page = await doCachedLogin(browser, { + user: adminuser, + url: 'part/76/history' + }); + + await page.getByText('widget.purple | Purple Widget').first().waitFor(); + + // The history of the part includes stock tracking events for its stock items + await page.getByText('Shipped against Sales Order').first().waitFor(); + await page.getByText('Customer D').first().waitFor(); + await page.getByText('SO0009').first().waitFor(); + await page.getByText('Installed into assembly').first().waitFor(); + await page.getByText('Purple Widget #35').first().waitFor(); + + // ...as well as changes to the part itself (e.g. parameters) + await page.getByText('Color: Purple').first().waitFor(); +}); + +test('Stock - History', async ({ browser }) => { + // Note: The 'admin' user has the "Show Change History" setting enabled + const page = await doCachedLogin(browser, { + user: adminuser, + url: 'stock/item/176/details' + }); await page.getByRole('link', { name: 'Widget Assembly # 2' }).waitFor(); - // Navigate to the "stock tracking" tab - await loadTab(page, 'Stock Tracking'); - - await page.getByText('Factory/Office Block/Room').first().waitFor(); - await page.getByRole('link', { name: 'Widget Assembly' }).waitFor(); - await page.getByRole('cell', { name: 'Installed into assembly' }).waitFor(); - - /* Add some more stock items and tracking information: + /* Add some more stock items and history information: * - Duplicate this stock item * - Give it a unique serial number - * - Ensure the tracking information is duplicated correctly + * - Ensure the creation of the new item is recorded * - Delete the new stock item - * - Ensure that the tracking information is retained against the base part + * - Ensure that the history is retained against the base part */ // Duplicate the stock item @@ -540,13 +568,10 @@ test('Stock - Tracking', async ({ browser }) => { .fill('9876'); await page.getByRole('button', { name: 'Submit' }).click(); - // Check stock tracking information is correct + // Check that the creation of the new item is recorded await page.getByText('Serial Number: 9876').first().waitFor(); - await loadTab(page, 'Stock Tracking'); - await page - .getByRole('cell', { name: 'Stock item created' }) - .first() - .waitFor(); + await loadTab(page, 'History', true); + await page.getByText('Stock item created').first().waitFor(); // Delete this stock item await page @@ -557,17 +582,10 @@ test('Stock - Tracking', async ({ browser }) => { .click(); await page.getByRole('button', { name: 'Delete' }).click(); - // Check stock tracking for base part - await loadTab(page, 'Stock History'); - await page.getByRole('button', { name: 'Stock Tracking' }).click(); - - await page.getByText('Stock item no longer exists').first().waitFor(); - await page - .getByRole('cell', { name: 'Thumbnail Blue Widget' }) - .first() - .waitFor(); - - await page.getByText('# 162').first().waitFor(); + // The history of the deleted item is retained against the base part + await loadTab(page, 'History', true); + await page.getByText('Object was deleted').first().waitFor(); + await page.getByText('Stock item created').first().waitFor(); }); test('Stock - Location', async ({ browser }) => {