Reference fields (#3267)

* Adds a configurable 'reference pattern' to the IndexingReferenceMixin class

* Expand tests for reference_pattern validator:

- Prevent inclusion of illegal characters
- Prevent multiple groups of hash (#) characters
- Add unit tests

* Validator now checks for valid strftime formatter

* Adds build order reference pattern

* Adds function for creating a valid regex from the supplied pattern

- More unit tests
- Use it to validate BuildOrder reference field

* Refactoring the whole thing again - try using python string.format

* remove datetime-matcher from requirements.txt

* Add some more formatting helper functions

- Construct a regular expression from a format string
- Extract named values from a string, based on a format string

* Fix validator for build order reference field

* Adding unit tests for the new format string functionality

* Adds validation for reference fields

* Require the 'ref' format key as part of a valid reference pattern

* Extend format extraction to allow specification of integer groups

* Remove unused import

* Fix requirements

* Add method for generating the 'next' reference field for a model

* Fix function for generating next BuildOrder reference value

- A function is required as class methods cannot be used
- Simply wraps the existing class method

* Remove BUILDORDER_REFERENCE_REGEX setting

* Add unit test for build order reference field validation

* Adds unit testing for extracting integer values from a reference field

* Fix bugs from previous commit

* Add unit test for generation of default build order reference

* Add data migration for BuildOrder model

- Update reference field with old prefix
- Construct new pattern based on old prefix

* Adds unit test for data migration

- Check that the BuildOrder reference field is updated as expected

* Remove 'BUILDORDER_REFERENCE_PREFIX' setting

* Adds new setting for SalesOrder reference pattern

* Update method by which next reference value is generated

* Improved error handling in api_tester code

* Improve automated generation of order reference fields

- Handle potential errors
- Return previous reference if something goes wrong

* SalesOrder reference has now been updated also

- New reference pattern setting
- Updated default and validator for reference field
- Updated serializer and API
- Added unit tests

* Migrate the "PurchaseOrder" reference field to the new system

* Data migration for SalesOrder and PurchaseOrder reference fields

* Remove PURCHASEORDER_REFERENCE_PREFIX

* Remove references to SALESORDER_REFERENCE_PREFIX

* Re-add maximum value validation

* Bug fixes

* Improve algorithm for generating new reference

- Handle case where most recent reference does not conform to the reference pattern

* Fixes for 'order' unit tests

* Unit test fixes for order app

* More unit test fixes

* More unit test fixing

* Revert behaviour for "extract_int" clipping function

* Unit test value fix

* Prevent build order notification if we are importing records

InvenTree/build/test_migrations.py

@@ -104,3 +104,57 @@ class TestReferenceMigration(MigratorTestCase):
         # Check that the build reference is properly assigned
         for build in Build.objects.all():
             self.assertEqual(str(build.reference), str(build.pk))
+
+
+class TestReferencePatternMigration(MigratorTestCase):
+    """Unit test for data migration which converts reference to new format.
+
+    Ref: https://github.com/inventree/InvenTree/pull/3267
+    """
+
+    migrate_from = ('build', '0019_auto_20201019_1302')
+    migrate_to = ('build', helpers.getNewestMigrationFile('build'))
+
+    def prepare(self):
+        """Create some initial data prior to migration"""
+
+        Setting = self.old_state.apps.get_model('common', 'inventreesetting')
+
+        # Create a custom existing prefix so we can confirm the operation is working
+        Setting.objects.create(
+            key='BUILDORDER_REFERENCE_PREFIX',
+            value='BuildOrder-',
+        )
+
+        Part = self.old_state.apps.get_model('part', 'part')
+
+        assembly = Part.objects.create(
+            name='Assy 1',
+            description='An assembly',
+            level=0, lft=0, rght=0, tree_id=0,
+        )
+
+        Build = self.old_state.apps.get_model('build', 'build')
+
+        for idx in range(1, 11):
+            Build.objects.create(
+                part=assembly,
+                title=f"Build {idx}",
+                quantity=idx,
+                reference=f"{idx + 100}",
+                level=0, lft=0, rght=0, tree_id=0,
+            )
+
+    def test_reference_migration(self):
+        """Test that the reference fields have been correctly updated"""
+
+        Build = self.new_state.apps.get_model('build', 'build')
+
+        for build in Build.objects.all():
+            self.assertTrue(build.reference.startswith('BuildOrder-'))
+
+        Setting = self.new_state.apps.get_model('common', 'inventreesetting')
+
+        pattern = Setting.objects.get(key='BUILDORDER_REFERENCE_PATTERN')
+
+        self.assertEqual(pattern.value, 'BuildOrder-{ref:04d}')