inventree/InvenTree
2
0
Fork 0
mirror of https://github.com/inventree/InvenTree.git synced 2025-06-12 18:15:40 +00:00
Code Activity

Code structure refactor (#5582)

Browse Source 
* moved docker files to /contrib/container

* changed code owners to make more precise

* updated CI to use new subdirs

* added manual trigger for testing

* moved ci files

* moved assets into subdir

* moved deploy template file to contrib

* moved django files to src/backend

* updated paths in scripts etc

* updated reqs path

* fixed version file path

* fixed flake8 path

* fixed path to node ressources

* fixed task paths

* added dep path for node

* removed unused yarn lockfile

* removed unused ci script

* updated internal backend paths for tasks

* updated translation stats path

* fixed source path for coverage

* fixed main commit repo path

* fit in changes from testing

* gather packager improvements (#149)

* Matmair/issue5578 (#143)

* moved docker files to /contrib/container

* changed code owners to make more precise

* updated CI to use new subdirs

* added manual trigger for testing

* moved ci files

* moved assets into subdir

* moved deploy template file to contrib

* moved django files to src/backend

* updated paths in scripts etc

* updated reqs path

* fixed version file path

* fixed flake8 path

* fixed path to node ressources

* fixed task paths

* added dep path for node

* removed unused yarn lockfile

* removed unused ci script

* updated internal backend paths for tasks

* updated translation stats path

* fixed source path for coverage

* fixed main commit repo path

* fix docker path

* use project dir

* move project dir command

* fixed docker paths

* another fix?

* seperate tasks out

* remove tasks

* some debugging

* ci: add .deepsource.toml

* Update .deepsource.toml

* also ignore migrations

* more debugging

* fix path issues

* remove debug script

* fix style

* change locale path

* Fixed paths for requirements

* Added dummy requirements to fool packager

* fixed exec path

* remove deepsource

---------

Co-authored-by: deepsource-io[bot] <42547082+deepsource-io[bot]@users.noreply.github.com>

* Added docs for file structure

* Fixed style errors

* updated deepsource paths

* fix deepsource paths

* fixed reqs

* merge fixes

* move newly added dirs too

* fix reqs files

* another dep fix

* merge upstream/master

* revert removal of tags

* merge upstream

* enabled detection of old config files

* adapt coverage src

* also detect and support old location for plugins.txt

* style fix

* fix ~/init.sh location

* fix requirements path

* fix config to current master

* move new folders

* fix import order

* fix paths for qc_check

* fix docs build

* fix fix path

* set docker project dir

* just use a cd

* set image path?

* set file correct

* fix copy path

* fix tasks dir

* fix init path

* fix copy path

* set prject dir

* fix paths

* remove old prod files

* fix dev env path

* set docker file

* Fix devcontainer docker compose file

* fix login attempt values

* fix init.sh path

* Fix pathing for Docker

* Docker build fix

- Set INVENTREE_BACKEND_DIR separately

* Update init.sh

* Fix path

* Update requirements.txt

* merge

* fix rq merge

* fix docker compose usage

---------

Co-authored-by: deepsource-io[bot] <42547082+deepsource-io[bot]@users.noreply.github.com>
Co-authored-by: Oliver <oliver.henry.walters@gmail.com>
This commit is contained in:
Matthias Mair
2024-04-03 02:16:59 +01:00
committed by GitHub
parent dd889e8eeb
commit 0bace3f3af
3317 changed files with 532 additions and 1194 deletions
Download Patch File Download Diff File Expand all files Collapse all files

103
.github/scripts/check_js_templates.py vendored Normal file
View File

@ -0,0 +1,103 @@
"""Test that the "translated" javascript files to not contain template tags which need to be determined at "run time".
This is because the "translated" javascript files are compiled into the "static" directory.
They should only contain template tags that render static information.
"""
import os
import pathlib
import re
import sys
here = os.path.abspath(os.path.dirname(__file__))
template_dir = os.path.abspath(os.path.join(here, '..', 'InvenTree', 'templates'))
# We only care about the 'translated' files
js_i18n_dir = os.path.join(template_dir, 'js', 'translated')
js_dynamic_dir = os.path.join(template_dir, 'js', 'dynamic')
errors = 0
print('=================================')
print('Checking static javascript files:')
print('=================================')
def check_invalid_tag(data):
"""Check for invalid tags."""
pattern = r'{%(\w+)'
err_count = 0
for idx, line in enumerate(data):
results = re.findall(pattern, line)
for result in results:
err_count += 1
print(f' - Error on line {idx + 1}: %{{{result[0]}')
return err_count
def check_prohibited_tags(data):
"""Check for prohibited tags."""
allowed_tags = [
'if',
'elif',
'else',
'endif',
'for',
'endfor',
'trans',
'load',
'include',
'url',
]
pattern = r'{% (\w+)\s'
err_count = 0
for idx, line in enumerate(data):
for tag in re.findall(pattern, line):
if tag not in allowed_tags:
print(f" > Line {idx + 1} contains prohibited template tag '{tag}'")
err_count += 1
return err_count
for filename in pathlib.Path(js_i18n_dir).rglob('*.js'):
print(f"Checking file 'translated/{os.path.basename(filename)}':")
with open(filename, 'r') as js_file:
data = js_file.readlines()
errors += check_invalid_tag(data)
errors += check_prohibited_tags(data)
for filename in pathlib.Path(js_dynamic_dir).rglob('*.js'):
print(f"Checking file 'dynamic/{os.path.basename(filename)}':")
# Check that the 'dynamic' files do not contains any translated strings
with open(filename, 'r') as js_file:
data = js_file.readlines()
invalid_tags = ['blocktrans', 'blocktranslate', 'trans', 'translate']
err_count = 0
for idx, line in enumerate(data):
for tag in invalid_tags:
tag = '{% ' + tag
if tag in line:
err_count += 1
print(f" > Error on line {idx + 1}: Prohibited tag '{tag}' found")
if errors > 0:
print(f'Found {errors} incorrect template tags')
sys.exit(errors)

28
.github/scripts/check_migration_files.py vendored Normal file
View File

@ -0,0 +1,28 @@
"""Check that there are no database migration files which have not been committed."""
import subprocess
import sys
print('Checking for unstaged migration files...')
cmd = ['git', 'ls-files', '--exclude-standard', '--others']
proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = proc.communicate()
migrations = []
for line in str(out.decode()).split('\n'):
if '/migrations/' in line:
migrations.append(line)
if len(migrations) == 0:
sys.exit(0)
print('There are {n} unstaged migration files:'.format(n=len(migrations)))
for m in migrations:
print(' - {m}'.format(m=m))
sys.exit(len(migrations))

196
.github/scripts/version_check.py vendored Normal file
View File

@ -0,0 +1,196 @@
"""Ensure that the release tag matches the InvenTree version number.
Behaviour:
master / main branch:
- version number must end with 'dev'
tagged branch:
- version number must match tag being built
- version number cannot already exist as a release tag
"""
import json
import os
import re
import sys
from pathlib import Path
import requests
def get_existing_release_tags():
"""Request information on existing releases via the GitHub API."""
# Check for github token
token = os.getenv('GITHUB_TOKEN', None)
headers = None
if token:
headers = {'Authorization': f'Bearer {token}'}
response = requests.get(
'https://api.github.com/repos/inventree/inventree/releases', headers=headers
)
if response.status_code != 200:
raise ValueError(
f'Unexpected status code from GitHub API: {response.status_code}'
)
data = json.loads(response.text)
# Return a list of all tags
tags = []
for release in data:
tag = release['tag_name'].strip()
match = re.match(r'^.*(\d+)\.(\d+)\.(\d+).*$', tag)
if len(match.groups()) != 3:
print(f"Version '{tag}' did not match expected pattern")
continue
tags.append([int(x) for x in match.groups()])
return tags
def check_version_number(version_string, allow_duplicate=False):
"""Check the provided version number.
Returns True if the provided version is the 'newest' InvenTree release
"""
print(f"Checking version '{version_string}'")
# Check that the version string matches the required format
match = re.match(r'^(\d+)\.(\d+)\.(\d+)(?: dev)?$', version_string)
if not match or len(match.groups()) != 3:
raise ValueError(
f"Version string '{version_string}' did not match required pattern"
)
version_tuple = [int(x) for x in match.groups()]
# Look through the existing releases
existing = get_existing_release_tags()
# Assume that this is the highest release, unless told otherwise
highest_release = True
for release in existing:
if release == version_tuple and not allow_duplicate:
raise ValueError(f"Duplicate release '{version_string}' exists!")
if release > version_tuple:
highest_release = False
print(f'Found newer release: {str(release)}')
return highest_release
if __name__ == '__main__':
if 'only_version' in sys.argv:
here = Path(__file__).parent.absolute()
version_file = here.joinpath('..', 'InvenTree', 'InvenTree', 'api_version.py')
text = version_file.read_text()
results = re.findall(r"""INVENTREE_API_VERSION = (.*)""", text)
print(results[0])
exit(0)
# GITHUB_REF_TYPE may be either 'branch' or 'tag'
GITHUB_REF_TYPE = os.environ['GITHUB_REF_TYPE']
# GITHUB_REF may be either 'refs/heads/<branch>' or 'refs/heads/<tag>'
GITHUB_REF = os.environ['GITHUB_REF']
GITHUB_REF_NAME = os.environ['GITHUB_REF_NAME']
GITHUB_BASE_REF = os.environ['GITHUB_BASE_REF']
# Print out version information, makes debugging actions *much* easier!
print(f'GITHUB_REF: {GITHUB_REF}')
print(f'GITHUB_REF_NAME: {GITHUB_REF_NAME}')
print(f'GITHUB_REF_TYPE: {GITHUB_REF_TYPE}')
print(f'GITHUB_BASE_REF: {GITHUB_BASE_REF}')
here = Path(__file__).parent.absolute()
version_file = here.joinpath(
'..', '..', 'src', 'backend', 'InvenTree', 'InvenTree', 'version.py'
)
version = None
with open(version_file, 'r') as f:
text = f.read()
# Extract the InvenTree software version
results = re.findall(r"""INVENTREE_SW_VERSION = '(.*)'""", text)
if len(results) != 1:
print(f'Could not find INVENTREE_SW_VERSION in {version_file}')
sys.exit(1)
version = results[0]
print(f"InvenTree Version: '{version}'")
# Check version number and look for existing versions
# If a release is found which matches the current tag, throw an error
allow_duplicate = False
# Note: on a 'tag' (release) we *must* allow duplicate versions, as this *is* the version that has just been released
if GITHUB_REF_TYPE == 'tag':
allow_duplicate = True
# Note: on a push to 'stable' branch we also allow duplicates
if GITHUB_BASE_REF == 'stable':
allow_duplicate = True
highest_release = check_version_number(version, allow_duplicate=allow_duplicate)
# Determine which docker tag we are going to use
docker_tags = None
if GITHUB_REF_TYPE == 'tag':
# GITHUB_REF should be of the form /refs/heads/<tag>
version_tag = GITHUB_REF.split('/')[-1]
print(f"Checking requirements for tagged release - '{version_tag}':")
if version_tag != version:
print(f"Version number '{version}' does not match tag '{version_tag}'")
sys.exit
if highest_release:
docker_tags = [version_tag, 'stable']
else:
docker_tags = [version_tag]
elif GITHUB_REF_TYPE == 'branch':
# Otherwise we know we are targeting the 'master' branch
docker_tags = ['latest']
else:
print('Unsupported branch / version combination:')
print(f'InvenTree Version: {version}')
print('GITHUB_REF_TYPE:', GITHUB_REF_TYPE)
print('GITHUB_BASE_REF:', GITHUB_BASE_REF)
print('GITHUB_REF:', GITHUB_REF)
sys.exit(1)
if docker_tags is None:
print('Docker tags could not be determined')
sys.exit(1)
print(f"Version check passed for '{version}'!")
print(f"Docker tags: '{docker_tags}'")
# Ref: https://getridbug.com/python/how-to-set-environment-variables-in-github-actions-using-python/
with open(os.getenv('GITHUB_ENV'), 'a') as env_file:
# Construct tag string
tags = ','.join([f'inventree/inventree:{tag}' for tag in docker_tags])
env_file.write(f'docker_tags={tags}\n')
if GITHUB_REF_TYPE == 'tag' and highest_release:
env_file.write('stable_release=true\n')