mirror of
https://github.com/inventree/InvenTree.git
synced 2025-06-16 12:05:53 +00:00
[Feature] Data export plugins (#9096)
* Move data export code out of "importer" directory * Refactoring to allow data export via plugin * Add brief docs framework * Add basic DataExportMixin class * Pass context data through to the serializer * Extract custom serializer * Refactoring * Add builtin plugin for BomExport * More refactoring * Cleanup for UseForm hooks * Allow GET methods in forms * Create new 'exporter' app * Refactor imports * Run cleanup task on boot * Add enumeration for plugin mixin types * Refactor with_mixin call * Generate export options serializer * Pass plugin information through * Offload export functionality to the plugin * Generate output * Download generated file * Refactor frontend code * Generate params for downloading * Pass custom fields through to the plugin * Implement multi-level export for BOM data * Export supplier and manufacturer information * Export substitute data * Remove old BOM exporter * Export part parameter data * Try different app order * Use GET instead of POST request - Less 'dangerous' - no chance of performing a destructive operation * Fix for constructing query parameters - Ignore any undefined values! * Trying something * Revert to POST - Required, other query data are ignored * Fix spelling mistakes * Remove SettingsMixin * Revert python version * Fix for settings.py * Fix missing return * Fix for label mixin code * Run playwright tests in --host mode * Fix for choice field - Prevent empty value if field is required * Remove debug prints * Update table header * Playwright tests for data export * Rename app from "exporter" to "data_exporter" * Add frontend table for export sessions * Updated playwright testing * Fix for unit test * Fix build order unit test * Back to using GET instead of POST - Otherwise, users need POST permissions to export! - A bit of trickery with the forms architecture * Fix remaining unit tests * Implement unit test for BOM export - Including test for custom plugin * Fix unit test * Bump API version * Enhanced playwright tests * Add debug for CI testing * Single unit test only (for debugging) * Fix typo * typo fix * Remove debugs * Docs updates * Revert typo * Update tests * Serializer fix * Fix typo * Offload data export to the background worker - Requires mocking the original request object - Will need some further unit testing! * Refactor existing models into DataOutput - Remove LabelOutput table - Remove ReportOutput table - Remove ExportOutput table - Consolidate into single API endpoint * Remove "output" tables from frontend * Refactor frontend hook to be generic * Frontend now works with background data export * Fix tasks.py * Adjust unit tests * Revert 'plugin_key' to 'plugin' * Improve user checking when printing * Updates * Remove erroneous migration file * Tweak plugin registry * Adjust playwright tests * Refactor data export - Convert into custom hook - Enable for calendar view also * Add playwright tests * Adjust unit testing * Tweak unit tests * Add extra timeout to data export * Fix for RUF045
This commit is contained in:
Oliver
GitHub
@ -7,7 +7,7 @@ The API schema as documented below is generated using the [drf-spectactular](htt
|
||||
|
||||
## API Version
|
||||
|
||||
This documentation is for API version: `171`
|
||||
This documentation is for API version: `315`
|
||||
|
||||
!!! tip "API Schema History"
|
||||
We track API schema changes, and provide a snapshot of each API schema version in the [API schema repository](https://github.com/inventree/schema/).
|
||||
|
||||
2
docs/docs/build/bom.md
vendored
2
docs/docs/build/bom.md
vendored
@ -119,7 +119,7 @@ Select a part in the list and click on "Add Substitute" button to confirm.
|
||||
|
||||
### Validate BOM
|
||||
|
||||
After [adding BOM items manually](#add-bom-item) or [uploading a BOM file](./bom_import.md), you should see the following view:
|
||||
After [adding BOM items manually](#add-bom-item) or uploading a BOM file, you should see the following view:
|
||||
{% with id="bom_invalid", url="build/bom_invalid.png", description="Invalid BOM View" %}
|
||||
{% include 'img.html' %}
|
||||
{% endwith %}
|
||||
|
||||
44
docs/docs/build/bom_export.md
vendored
44
docs/docs/build/bom_export.md
vendored
@ -1,44 +0,0 @@
|
||||
---
|
||||
title: BOM Export
|
||||
---
|
||||
|
||||
## Exporting BOM Data
|
||||
|
||||
|
||||
BOM data can be exported for any given assembly by selecting the *Export BOM* action from the BOM actions menu.
|
||||
|
||||
You will be presented with the *Export BOM* options dialog, shown below:
|
||||
|
||||
{% with id="bom_export", url="build/bom_export.png", description="Export BOM Data" %}
|
||||
{% include 'img.html' %}
|
||||
{% endwith %}
|
||||
|
||||
### BOM Export Options
|
||||
|
||||
**Format**
|
||||
|
||||
Select the file format for the exported BOM data
|
||||
|
||||
**Multi Level BOM**
|
||||
|
||||
If selected, BOM data will be included for any subassemblies. If not selected, only top level (flat) BOM data will be exported.
|
||||
|
||||
**Levels**
|
||||
|
||||
Define the maximum level of data to export for subassemblies. If set to zero, all levels of subassembly data will be exported.
|
||||
|
||||
**Include Parameter Data**
|
||||
|
||||
Include part parameter data in the exported dataset.
|
||||
|
||||
**Include Stock Data**
|
||||
|
||||
Include part stock level information in the exported dataset.
|
||||
|
||||
**Include Manufacturer Data**
|
||||
|
||||
Include part manufacturer information in the exported dataset.
|
||||
|
||||
**Include Supplier Data**
|
||||
|
||||
Include part supplier information in the exported dataset.
|
||||
49
docs/docs/build/bom_import.md
vendored
49
docs/docs/build/bom_import.md
vendored
@ -1,49 +0,0 @@
|
||||
---
|
||||
title: BOM Import
|
||||
---
|
||||
|
||||
## Importing BOM Data
|
||||
|
||||
Uploading a BOM to InvenTree is a three steps process:
|
||||
|
||||
1. Upload BOM file
|
||||
0. Select matching InvenTree fields
|
||||
0. Select matching InvenTree parts.
|
||||
|
||||
To upload a BOM file, navigate to the part/assembly detail page then click on the "BOM" tab. On top of the tab view, click on the <span class='fas fa-edit'></span> icon then, after the page reloads, click on the <span class='fas fa-file-upload'></span> icon.
|
||||
|
||||
The following view will load:
|
||||
{% with id="bom_upload_file", url="build/bom_upload_file.png", description="BOM Upload View" %}
|
||||
{% include 'img.html' %}
|
||||
{% endwith %}
|
||||
|
||||
#### Upload BOM File
|
||||
|
||||
Click on the "Choose File" button, select your BOM file when prompted then click on the "Upload File" button.
|
||||
|
||||
!!! info "BOM Formats"
|
||||
The following BOM file formats are supported: CSV, TSV, XLS, XLSX, JSON and YAML
|
||||
|
||||
#### Select Fields
|
||||
|
||||
Once the BOM file is uploaded, the following view will load:
|
||||
{% with id="bom_select_fields", url="build/bom_select_fields.png", description="Select Fields View" %}
|
||||
{% include 'img.html' %}
|
||||
{% endwith %}
|
||||
|
||||
InvenTree will attempt to automatically match the BOM file columns with InvenTree part fields. `Part_Name` is a **required** field for the upload process and moving on to the next step. Specifying the `Part_IPN` field matching is very powerful as it allows to create direct pointers to InvenTree parts.
|
||||
|
||||
Once you have selected the corresponding InvenTree fields, click on the "Submit Selections" button to move on to the next step.
|
||||
|
||||
#### Select Parts
|
||||
|
||||
Once the BOM file columns and InvenTree fields are correctly matched, the following view will load:
|
||||
{% with id="bom_select_parts", url="build/bom_select_parts.png", description="Select Parts View" %}
|
||||
{% include 'img.html' %}
|
||||
{% endwith %}
|
||||
|
||||
InvenTree automatically tries to match parts from the BOM file with parts in its database. For parts that are found in InvenTree's database, the `Select Part` field selection will automatically point to the matching database part.
|
||||
|
||||
In this view, you can also edit the parts `Reference` and `Quantity` fields.
|
||||
|
||||
Once you have selected the corresponding InvenTree parts, click on the "Submit BOM" button to complete the BOM upload process.
|
||||
@ -113,6 +113,7 @@ Supported mixin classes are:
|
||||
| [AppMixin](./plugins/app.md) | Integrate additional database tables |
|
||||
| [BarcodeMixin](./plugins/barcode.md) | Support custom barcode actions |
|
||||
| [CurrencyExchangeMixin](./plugins/currency.md) | Custom interfaces for currency exchange rates |
|
||||
| [DataExport](./plugins/export.md) | Customize data export functionality |
|
||||
| [EventMixin](./plugins/event.md) | Respond to events |
|
||||
| [LabelPrintingMixin](./plugins/label.md) | Custom label printing support |
|
||||
| [LocateMixin](./plugins/locate.md) | Locate and identify stock items |
|
||||
|
||||
122
docs/docs/extend/plugins/export.md
Normal file
122
docs/docs/extend/plugins/export.md
Normal file
@ -0,0 +1,122 @@
|
||||
---
|
||||
title: Data Export Mixin
|
||||
---
|
||||
|
||||
## DataExportMixin
|
||||
|
||||
The `DataExportMixin` class provides a plugin with the ability to customize the data export process. The [InvenTree API](../../api/api.md) provides an integrated method to export a dataset to a tabulated file. The default export process is generic, and simply exports the data presented via the API in a tabulated file format.
|
||||
|
||||
Custom data export plugins allow this process to be adjusted:
|
||||
|
||||
- Data columns can be added or removed
|
||||
- Rows can be removed or added
|
||||
- Custom calculations or annotations can be performed.
|
||||
|
||||
### Supported Export Types
|
||||
|
||||
Each plugin can dictate which datasets are supported using the `supports_export` method. This allows a plugin to dynamically specify whether it can be selected by the user for a given export session.
|
||||
|
||||
::: plugin.base.integration.DataExport.DataExportMixin.supports_export
|
||||
options:
|
||||
show_bases: False
|
||||
show_root_heading: False
|
||||
show_root_toc_entry: False
|
||||
show_sources: True
|
||||
summary: False
|
||||
members: []
|
||||
|
||||
The default implementation returns `True` for all data types.
|
||||
|
||||
### Filename Generation
|
||||
|
||||
The `generate_filename` method constructs a filename for the exported file.
|
||||
|
||||
::: plugin.base.integration.DataExport.DataExportMixin.generate_filename
|
||||
options:
|
||||
show_bases: False
|
||||
show_root_heading: False
|
||||
show_root_toc_entry: False
|
||||
show_sources: True
|
||||
summary: False
|
||||
members: []
|
||||
|
||||
### Adjust Columns
|
||||
|
||||
The `update_headers` method allows the plugin to adjust the columns selected to be exported to the file.
|
||||
|
||||
::: plugin.base.integration.DataExport.DataExportMixin.update_headers
|
||||
options:
|
||||
show_bases: False
|
||||
show_root_heading: False
|
||||
show_root_toc_entry: False
|
||||
show_sources: True
|
||||
summary: False
|
||||
members: []
|
||||
|
||||
### Queryset Filtering
|
||||
|
||||
The `filter_queryset` method allows the plugin to provide custom filtering to the database query, before it is exported.
|
||||
|
||||
::: plugin.base.integration.DataExport.DataExportMixin.filter_queryset
|
||||
options:
|
||||
show_bases: False
|
||||
show_root_heading: False
|
||||
show_root_toc_entry: False
|
||||
show_sources: True
|
||||
summary: False
|
||||
members: []
|
||||
|
||||
### Export Data
|
||||
|
||||
The `export_data` method performs the step of transforming a [Django QuerySet]({% include "django.html" %}/ref/models/querysets/) into a dataset which can be processed by the [tablib](https://tablib.readthedocs.io/en/stable/) library.
|
||||
|
||||
::: plugin.base.integration.DataExport.DataExportMixin.export_data
|
||||
options:
|
||||
show_bases: False
|
||||
show_root_heading: False
|
||||
show_root_toc_entry: False
|
||||
show_sources: True
|
||||
summary: False
|
||||
members: []
|
||||
|
||||
Note that the default implementation simply uses the builtin tabulation functionality of the provided serializer class. In most cases, this will be sufficient.
|
||||
|
||||
## Custom Export Options
|
||||
|
||||
To provide the user with custom options to control the behavior of the export process *at the time of export*, the plugin can define a custom serializer class.
|
||||
|
||||
To enable this feature, define an `ExportOptionsSerializer` attribute on the plugin class which points to a DRF serializer class. Refer to the examples below for more information.
|
||||
|
||||
### Builtin Exporter Classes
|
||||
|
||||
InvenTree provides the following builtin data exporter classes.
|
||||
|
||||
### InvenTreeExporter
|
||||
|
||||
A generic exporter class which simply serializes the API output into a data file.
|
||||
|
||||
::: plugin.builtin.exporter.inventree_exporter.InvenTreeExporter
|
||||
options:
|
||||
show_bases: False
|
||||
show_root_heading: False
|
||||
show_root_toc_entry: False
|
||||
show_source: True
|
||||
members: []
|
||||
|
||||
### BOM Exporter
|
||||
|
||||
A custom exporter which only supports [bill of materials](../../build/bom.md) exporting.
|
||||
|
||||
::: plugin.builtin.exporter.bom_exporter.BomExporterPlugin
|
||||
options:
|
||||
show_bases: False
|
||||
show_root_heading: False
|
||||
show_root_toc_entry: False
|
||||
show_source: True
|
||||
members: []
|
||||
|
||||
## Source Code
|
||||
|
||||
The full source code of the `DataExportMixin` class:
|
||||
|
||||
{{ includefile("src/backend/InvenTree/plugin/base/integration/DataExport.py", title="DataExportMixin") }}
|
||||
@ -4,7 +4,7 @@ title: Report Mixin
|
||||
|
||||
## ReportMixin
|
||||
|
||||
The ReportMixin class provides a plugin with the ability to extend the functionality of custom [report templates](../../report/report.md). A plugin which implements the ReportMixin mixin class can add custom context data to a report template for rendering.
|
||||
The `ReportMixin` class provides a plugin with the ability to extend the functionality of custom [report templates](../../report/report.md). A plugin which implements the ReportMixin mixin class can add custom context data to a report template for rendering.
|
||||
|
||||
### Add Report Context
|
||||
|
||||
|
||||
@ -102,7 +102,7 @@ def check_link(url) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def get_build_enviroment() -> str:
|
||||
def get_build_environment() -> str:
|
||||
"""Returns the branch we are currently building on, based on the environment variables of the various CI platforms."""
|
||||
# Check if we are in ReadTheDocs
|
||||
if os.environ.get('READTHEDOCS') == 'True':
|
||||
@ -134,7 +134,7 @@ def define_env(env):
|
||||
- FileNotFoundError: If the directory does not exist, or the generated URL is invalid
|
||||
"""
|
||||
if branch == None:
|
||||
branch = get_build_enviroment()
|
||||
branch = get_build_environment()
|
||||
|
||||
if dirname.startswith('/'):
|
||||
dirname = dirname[1:]
|
||||
@ -173,7 +173,7 @@ def define_env(env):
|
||||
- FileNotFoundError: If the file does not exist, or the generated URL is invalid
|
||||
"""
|
||||
if branch == None:
|
||||
branch = get_build_enviroment()
|
||||
branch = get_build_environment()
|
||||
|
||||
if filename.startswith('/'):
|
||||
filename = filename[1:]
|
||||
|
||||
@ -130,8 +130,6 @@ nav:
|
||||
- Allocating Stock: build/allocate.md
|
||||
- Example Build Order: build/example.md
|
||||
- Bill of Materials: build/bom.md
|
||||
- Importing BOM Data: build/bom_import.md
|
||||
- Exporting BOM Data: build/bom_export.md
|
||||
- Orders:
|
||||
- Companies: order/company.md
|
||||
- Purchase Orders: order/purchase_order.md
|
||||
@ -208,6 +206,7 @@ nav:
|
||||
- App Mixin: extend/plugins/app.md
|
||||
- Barcode Mixin: extend/plugins/barcode.md
|
||||
- Currency Mixin: extend/plugins/currency.md
|
||||
- Data Export Mixin: extend/plugins/export.md
|
||||
- Event Mixin: extend/plugins/event.md
|
||||
- Icon Pack Mixin: extend/plugins/icon.md
|
||||
- Label Printing Mixin: extend/plugins/label.md
|
||||
|
||||
Reference in New Issue
Block a user