pax_global_header00006660000000000000000000000064152217666040014523gustar00rootroot0000000000000052 comment=e159b66d38c5b87790403604b339a3f1829afb30 decopy-0.3.2/000077500000000000000000000000001522176660400130105ustar00rootroot00000000000000decopy-0.3.2/.coveragerc000066400000000000000000000001621522176660400151300ustar00rootroot00000000000000; vim: ft=dosini [paths] source = decopy/ [run] branch = True source = decopy/ omit = decopy/__init__.py decopy-0.3.2/.gitignore000066400000000000000000000000531522176660400147760ustar00rootroot00000000000000.pdm-python *.pyc .gdb_history __pycache__ decopy-0.3.2/COPYING000066400000000000000000000013721522176660400140460ustar00rootroot00000000000000Copyright: 2016, Maximiliano Curia License: ISC Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. . THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. decopy-0.3.2/INSTALL.md000066400000000000000000000014071522176660400144420ustar00rootroot00000000000000# Installation It's recommended that you use the debian package, but if you want to use this outside of the package, you will need the following pieces: ## Required python modules * python3-xdg * python3-debian ## Required Software * exiftool: to handle the image files * bzcat: to handle the bzipped files * zcat: to handle the gzipped files * xzcat: to handle the xzipped files ## Recommended python modules * python3-tqdm: used for the --progress option * python3-regex: allows faster regular expression matching * python3-zipfile-zstd: allows checking zip files that use the zstd compresison method * python3-ahocorasick: allows prefiltering, which is a significant performance improvement * python3-pathspec: honours .gitignore for the --gitignore option decopy-0.3.2/README.md000066400000000000000000000230331522176660400142700ustar00rootroot00000000000000decopy(1) -- Automatic debian/copyright generator ================================================= ## SYNTAX `decopy` [options] [files or dirs] ## DESCRIPTION Decopy is a tool that automates creating and updating the debian/copyright files. It does this by parsing all files in a source tree, figuring out the copyright attribution and license for each file (when possible) and then structuring the output in the copyright format 1.0. ## USAGE Simply running `decopy` in the source tree that needs to be parsed will process all files in the tree, group them by license, wildcard those groups and generate the proper output to be written to the debian/copyright file. If the source tree already includes a debian/copyright file, it will be used as input for files where the license could not be inferred, for the copyright of the debian directory and for the license snippets. The license texts for well known licenses are filled in automatically (see **--fill-licenses**); any that are not recognised need to be filled in manually. The files or directories specified as positional arguments can be used to limit the processing to the indicated subset instead of processing the complete source tree. ## EXAMPLES On top of the simple `decopy` usage, these examples show some of the most common usecases. Please see the following sections for detailed explanations of what each of the parameters do. * `decopy --root src/hello-1.0/ --output src/hello-1.0/debian/copyright`: This runs the parser in the hello-1.0 source tree and generates the debian/copyright file in the source tree (the debian/ directory needs to pre-exist this call). * `decopy --mode partial src/3rdparty/ninja`: This shows the licensing and copyright information for the `src/3rdparty/ninja` directory. Only that directory is analyzed and the output includes only files contained in it. * `decopy --group-by copyright`: This processes the complete current source tree, paragraphs are split not only according to the licenses but also according to the copyright owners. * `decopy src`: This will process the src directory but, provided debian/copyright already exists, show the complete updated debian/copyright file. ## COMMON OPTIONS * `--mode `: Indicates the mode to use when generating the output. The **full** mode (default) will generate a complete debian/copyright file, even if only some files were processed (the rest of the contents are taken from the original debian/copyright file). The **partial** mode will only print the copyright and licensing information for the indicated files. The **changed** mode restricts processing to the files that changed since the debian/copyright file was last committed to git (see **--changed** and **--since** below). * `--changed`: Shorthand for **--mode changed**. Scopes the run to the files that changed since the debian/copyright file was last committed. This requires the source tree to be a git repository; if the changed set cannot be computed (git is not available, the tree is not a repository, or the copyright file is untracked) decopy warns and falls back to a full scan. * `--since `: Baseline git ref for **--changed**. Defaults to the commit that last modified the debian/copyright file. * `--output `, `-o `: Store the generated output in the indicated file. This will overwrite the file if it already exists. * `--root `: Process the indicated path instead of the current directory. * `--spdx`, `--no-spdx`: When **--spdx** is given, license short names are rendered using their SPDX identifiers instead of the DEP-5 copyright format names. The default is **--no-spdx**, which uses the DEP-5 names. * `--fill-licenses`, `--no-fill-licenses`: When **--fill-licenses** (the default) is given, the **License:** stanzas in the generated output are filled in with the corresponding license text: a reference to `/usr/share/common-licenses` for the licenses shipped there, or the full text for other well known licenses. With **--no-fill-licenses** the license text is left as a `TODO` placeholder to be filled in manually. ## EXPLAIN MODE Instead of regenerating debian/copyright, decopy can report what is out of date in the existing file, which is useful as a check (e.g. in CI). * `--explain`: Report what is out of date in debian/copyright instead of regenerating it: stale Files paragraphs, patterns that no longer match any file, files whose license changed, new files that are not yet covered and copyright holders that have drifted. Nothing is written; decopy exits with status 1 if anything is stale and 0 if the file is up to date. ## GROUPING CRITERIA The generated output will always create separate paragraphs for groups of files with different licenses, additionally, it can also split paragraphs according to the following grouping options. * `--group-by `: When grouping by **license** (default), all copyright owners of different files with the same license are listed together. When grouping by **copyright**, each different group of copyright owners is listed separatedly (files that have the exact same list of copyright owners are listed together). * `--split-on-license`: When a subdirectory has its own license file (COPYING, LICENSE, etc), the subdirectory is given its own paragraph, even if it has the same license as the main directory. This is the default. * `--no-split-on-license`: Even if a subdirectory has its own license file, the contents are listed together with other files in the tree that have the same license. * `--split-debian`: Even if the contents of the debian/ subdirectory have the same license as the main content, split it onto a separate paragraph. This is the default. * `--no-split-debian`: If the contents of the debian/ subdirectory have the same license as the main content, they are listed together. ## ARCHIVE EXPANSION Archive files (tar, zip and similar) found in the source tree are expanded and their contents are parsed as if they were regular files, so that licenses and copyright holders embedded inside archives are detected. * `--expand-archives`, `--no-expand-archives`: Enable or disable expanding archive files into their contents. Expansion is enabled by default (**--expand-archives**). * `--max-archive-depth `: Maximum depth of nested archive expansion (archives contained inside other archives). Defaults to 1. * `--max-archive-member-size `: Skip archive members larger than this many bytes. Defaults to 10 MB. ## LOG LEVEL OPTIONS The default logging shows WARN and ERROR messages. It can be modified with the following flags: * `--debug`: Show DEBUG, INFO, WARN and ERROR messages. * `--verbose`, `-v`: Show INFO, WARN and ERROR messages. * `--quiet`, `-q`: Show only ERROR messages. ## LESS COMMON OPTIONS * `--copyright-file `: Specify a different copyright file to be used as a base for the processing. Defaults to `debian/copyright` * `--text`, `-a`: Parse all files as text files, instead of using the different parsers according to the file types (e.g. using exiftool for images). * `--jobs `, `-j `: Number of jobs to run in parallel while processing files. Use 0 (the default) to pick a value automatically based on the available CPUs. * `--glob`, `--no-glob`: Enable or disable the use of glob (wildcard) patterns in the `Files:` field of the generated output. Globbing is enabled by default (**--glob**). * `--progress`, `--no-progress`: Show or hide the progress bar while processing files. The progress bar is shown by default (**--progress**). * `--exclude `, `-X `: Exclude files from being parsed. The path can be specified as a regular expression that is matched against the full relative path for the file (e.g. `-X '.*\.jpg$'` to exclude all files that have the `.jpg` extension. or `-X doc/` to exclude all files inside the `doc/` directory). As a special case, `debian/copyright` and `debian/changelog` are always excluded.. * `--gitignore`, `--no-gitignore`: Honour the repository's ignore lists (`.gitignore` and `.git/info/exclude`) while walking the tree, so ignored files are not parsed. This is enabled by default (**--gitignore**); use **--no-gitignore** to parse ignored files too. This requires the `pathspec` module; without it the option has no effect. The **Files-Excluded** patterns from the header of an existing debian/copyright file are always honoured, regardless of this option. ## COMPARISON WITH OTHER HELPERS Decopy was inspired by previous copyright helpers like [copyright-helper by Modestas Vainius][1], licensecheck(1) by Jonas Smedegaard, or the copyright helper shipped with debmake(1), written by Osamu Aoki. [1]: https://anonscm.debian.org/cgit/users/modax/copyright-helper.git/ The main difference with all of them, is that decopy aims to detects as many licenses as possible, including many more that those in the other helpers, as well as the fact that it's capable of processing more than just plain text files. Additionally, while it is possible to use it for the creation of the copyright file, its main usecase is to keep the copyright file updated with any changes that are done to the source tree as time passes. ## AUTHORS Decopy was written by Maximiliano Curia. The debian packaging and this manpage were created by Margarita Manterola. ## COPYRIGHT Decopy and all the associated code and documentation are released under the ISC license. decopy-0.3.2/TODO.md000066400000000000000000000004121522176660400140740ustar00rootroot00000000000000## Things that still need to be done * [ ] files wildcarding: `foo.*`, for `foo.c` and `foo.h`, take into account `foo.m` (that might be under a different license and listed first). * [ ] Read the AUTHORS file and use it in a similar way as the COPYING file. decopy-0.3.2/bin/000077500000000000000000000000001522176660400135605ustar00rootroot00000000000000decopy-0.3.2/bin/decopy000077500000000000000000000020321522176660400147660ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. import pathlib import sys if __name__ == "__main__": dirname = pathlib.Path(__file__).resolve(strict=True).parent top = dirname.parent sys.path.insert(0, str(top)) from decopy import decopy sys.exit(decopy.main()) decopy-0.3.2/decopy/000077500000000000000000000000001522176660400142735ustar00rootroot00000000000000decopy-0.3.2/decopy/__init__.py000066400000000000000000000000001522176660400163720ustar00rootroot00000000000000decopy-0.3.2/decopy/cmdoptions.py000077500000000000000000000243251522176660400170350ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Command line option handling.""" from __future__ import annotations import argparse import collections.abc as abc import dataclasses import logging import os import pathlib import typing as t from decopy.common._regex import rex from decopy.common.core_types import Options from decopy.common.log import logger class Defaults: exclude_file_regex: str = r""" # Ignore general backup files ~$| # Ignore emacs recovery files \.\#.*$| # Ignore vi swap files \..*\.swp$| # Ignore baz-style junk files or directories ^,,| # File-names that should be ignored (never directories) ^(?:DEADJOE|\.cvsignore|\.arch-inventory|\.bzrignore|\.gitignore)$| # File or directory names that should be ignored ^(?:CVS|RCS|\.deps|\{arch\}|\.arch-ids|\.svn|\.hg|_darcs|\.git| \.shelf|_MTN|\.bzr(?:\.backup|tags)?)$ """ exclude_directory_regex: str = r""" # Ignore baz-style junk files or directories ^,,| # File or directory names that should be ignored ^(?:CVS|RCS|\.deps|\{arch\}|\.arch-ids|\.svn|\.hg|_darcs|\.git| \.pc|\.shelf|_MTN|\.bzr(?:\.backup|tags)?)$| # Python compiled files ^__pycache__$ """ # Exclude debian/copyright and debian/changelog by default exclude_special_regex: str = r""" ^debian/(copyright|changelog)$ """ # Empty exclude regexp, to be overridden by users exclude_fullname_regex: str = r""" ^$ """ # Top directory for processing files root: str = "." # Files/dirs to parse files: t.ClassVar[list[str]] = [] # Mode to work on: # partial: only process the files/dirs specified and generate the output # for this subset # full: (default) parse the files/dirs specified, and generate the # copyright information for the whole source tree mode: str = "full" # copyright 1.0 file, relative to the package dir copyright_file: str = "debian/copyright" # group by license or by copyright group_by: str = "license" # Split groups on sub dirs containing licenses split_on_license: bool = True # Split the debian/* file paragraph split_debian: bool = True # Treat all files as text text: bool = False # Try to use glob patterns in the File: output glob: bool = True # Progress bar progress: bool = True # Number of parallel jobs (0 for auto) jobs: int = 0 # output filename, empty for stdout output: str = "" # Defaults that the Options dataclass already declares (in core_types). The CLI # reads them from there so each such default lives in exactly one place; fields # Options requires (mode, root, etc.) keep their CLI default in Defaults above. _OPTION_DEFAULTS: dict[str, t.Any] = { field.name: field.default for field in dataclasses.fields(Options) if field.default is not dataclasses.MISSING } def _add_boolean_argument( parser: argparse.ArgumentParser, name: str, *, default: bool, dest: str | None = None, ) -> None: """Helper to add options --name --no-name and set the default""" if dest is None: dest = name.replace("-", "_") _ = parser.add_argument(f"--{name}", action="store_true", dest=dest) _ = parser.add_argument(f"--no-{name}", action="store_false", dest=dest) parser.set_defaults(**{dest: default}) def _filename(arg: str) -> str: """argparse type: accept a path only if it exists.""" if not pathlib.Path(arg).exists(): msg = f"{arg} does not exist" raise argparse.ArgumentTypeError(msg) return arg def _build_parser() -> argparse.ArgumentParser: """Assemble the argument parser. Defaults come from Defaults (CLI-only values) and _OPTION_DEFAULTS (values the Options dataclass owns).""" parser = argparse.ArgumentParser( description="License checker and copyright dep5 helper", fromfile_prefix_chars="@", ) _ = parser.add_argument("-X", "--exclude", default=Defaults.exclude_fullname_regex) _ = parser.add_argument( "--mode", choices=["full", "partial", "changed"], default=Defaults.mode, ) _ = parser.add_argument( "--changed", action="store_const", const="changed", dest="mode", help=( "shorthand for --mode=changed: scope to the files changed since " "the copyright file was last committed (needs a git repository)" ), ) _ = parser.add_argument( "--since", default=_OPTION_DEFAULTS["since"], help=( "baseline git ref for --changed (default: the commit that last " "modified the copyright file)" ), ) _ = parser.add_argument("--copyright-file", default=Defaults.copyright_file) _ = parser.add_argument("--debug", action="store_true") _ = parser.add_argument("-v", "--verbose", action="store_true") _ = parser.add_argument("-q", "--quiet", action="store_true") _ = parser.add_argument("-a", "--text", action="store_true") _ = parser.add_argument( "--explain", action="store_true", default=_OPTION_DEFAULTS["explain"], help=( "report what is out of date in debian/copyright instead of " "regenerating it; exits 1 if anything is stale" ), ) _ = parser.add_argument( "--group-by", choices=["license", "copyright"], default=Defaults.group_by, ) _add_boolean_argument(parser, "split-on-license", default=Defaults.split_on_license) _add_boolean_argument(parser, "split-debian", default=Defaults.split_debian) _add_boolean_argument( parser, "gitignore", default=_OPTION_DEFAULTS["use_gitignore"], dest="use_gitignore", ) _add_boolean_argument(parser, "glob", default=Defaults.glob) _add_boolean_argument( parser, "fill-licenses", default=_OPTION_DEFAULTS["fill_licenses"] ) _add_boolean_argument(parser, "progress", default=Defaults.progress) _add_boolean_argument(parser, "spdx", default=_OPTION_DEFAULTS["spdx"]) _add_boolean_argument( parser, "expand-archives", default=_OPTION_DEFAULTS["expand_archives"] ) _ = parser.add_argument( "--max-archive-depth", type=int, default=_OPTION_DEFAULTS["max_archive_depth"], help="Max depth of nested archive expansion (default: 1)", ) _ = parser.add_argument( "--max-archive-member-size", type=int, default=_OPTION_DEFAULTS["max_archive_member_size"], help="Skip archive members larger than this many bytes (default: 10 MB)", ) _ = parser.add_argument( "--jobs", "-j", type=int, default=Defaults.jobs, help="Number of jobs to run in parallel (0 for auto)", ) _ = parser.add_argument("-o", "--output", default=Defaults.output) _ = parser.add_argument("--root", default=Defaults.root, type=_filename) _ = parser.add_argument("files", nargs="*", default=Defaults.files, type=_filename) return parser def _configure_logging(namespace: argparse.Namespace) -> None: """Set the module logger level from the verbosity flags.""" if namespace.debug: level = logging.DEBUG elif namespace.verbose: level = logging.INFO elif namespace.quiet: level = logging.ERROR else: level = logging.WARNING logging.basicConfig(format="[%(levelname)s] %(message)s") logger.setLevel(level) def _compile_excludes(namespace: argparse.Namespace) -> None: """Compile the exclude patterns onto the namespace as the Options re fields.""" namespace.exclude_fullname_re = rex.compile( namespace.exclude, rex.IGNORECASE | rex.VERBOSE, ) namespace.exclude_special_re = rex.compile( Defaults.exclude_special_regex, rex.IGNORECASE | rex.VERBOSE, ) namespace.exclude_file_re = rex.compile(Defaults.exclude_file_regex, rex.VERBOSE) namespace.exclude_directory_re = rex.compile( Defaults.exclude_directory_regex, rex.VERBOSE, ) def _normalise_paths(namespace: argparse.Namespace) -> None: """Normalise the output sentinel and resolve root/files against the tree.""" # - can also be used to specify the stdout if namespace.output == "-": namespace.output = "" # Force an absolute path for root, and sanitize the user option # (see #855778, for problem caused by a trailing /) namespace.root = str(pathlib.Path(namespace.root).absolute()) # Transform positional arguments into paths that are relative to the root namespace.files = [os.path.relpath(i, namespace.root) for i in namespace.files] def _to_options(namespace: argparse.Namespace) -> Options: """Build Options from the namespace by matching field names. Every Options field name that the namespace carries is copied over; fields Options defaults and the CLI does not set (e.g. files_excluded) keep their dataclass default. Namespace-only attributes (verbose, quiet, exclude) are ignored. """ field_names = {field.name for field in dataclasses.fields(Options)} return Options( **{ name: getattr(namespace, name) for name in field_names if hasattr(namespace, name) } ) def process_options(args: abc.Sequence[str] | None = None) -> Options: parser = _build_parser() namespace = parser.parse_args(args) _compile_excludes(namespace) _normalise_paths(namespace) _configure_logging(namespace) return _to_options(namespace) decopy-0.3.2/decopy/common/000077500000000000000000000000001522176660400155635ustar00rootroot00000000000000decopy-0.3.2/decopy/common/__init__.py000066400000000000000000000000001522176660400176620ustar00rootroot00000000000000decopy-0.3.2/decopy/common/_regex.py000066400000000000000000000024001522176660400174020ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Single source of truth for the regex backend. Use the third-party ``regex`` module when it is installed (it is faster and supports a richer syntax), and transparently fall back to the standard-library ``re`` otherwise. Every other module imports ``rex`` from here instead of repeating the try/except dance. """ from __future__ import annotations try: import regex as rex except ImportError: import re as rex # type: ignore[no-redef] __all__ = ["rex"] decopy-0.3.2/decopy/common/core_types.py000066400000000000000000000142641522176660400203200ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Core data types for decopy""" from __future__ import annotations import dataclasses import typing as t from debian.copyright import LicenseParagraph from decopy.common._regex import rex # Constants for license identification UNKNOWN = "Unknown" UNKNOWN_COPYRIGHTED = "UnknownCopyrighted" @t.final class License: """License representation for copyright files.""" __slots__ = ("name", "stored") def __init__(self, name: str) -> None: self.name: str = name self.stored: LicenseParagraph | None = None @t.override def __str__(self) -> str: return self.name class LicenseRegistry: """A per-run store of License instances, keyed by name. One registry is created per run and shared between the consumers that read or write a license's stored text. """ def __init__(self) -> None: self._licenses: dict[str, License] = {} def get(self, name: str) -> License: license_ = self._licenses.get(name) if license_ is None: license_ = License(name) self._licenses[name] = license_ return license_ @t.final class YearRange: """Year range representation for copyright dates.""" __slots__ = ("high", "low") def __init__( self, low: str | int | None = None, high: str | int | None = None, ) -> None: low_value = 0 if low is None else int(low) high_value = low_value if high is None else int(high) if low_value > high_value: low_value, high_value = high_value, low_value self.low: int = low_value self.high: int = high_value def __contains__(self, year: str | int) -> bool: value = int(year) return self.low <= value <= self.high def add(self, year: str | int | None) -> YearRange: value = int(year if year is not None else 0) if not value: return self if not self.low or value < self.low: self.low = value if not self.high or self.high < value: self.high = value return self def newer(self, other: YearRange) -> bool: if self.high and other.high: return other.high > self.high return bool(not self.high and other.high) def merge(self, other: YearRange) -> YearRange: _ = self.add(other.low) _ = self.add(other.high) return self def __bool__(self) -> bool: return bool(self.low) @t.override def __str__(self) -> str: if not self.low: return "" if self.low == self.high: return str(self.low) return f"{self.low}-{self.high}" @t.final class CopyrightHolder: """Copyright holder representation for copyright files.""" __slots__ = ("email", "name", "years") def __init__(self, name: str, email: str, years: YearRange | None = None) -> None: self.name = name self.email = email self.years = years if years is not None else YearRange() def merge(self, other: CopyrightHolder) -> t.Self: if other.name and self.years.newer(other.years): self.name = other.name if not self.email and other.email: self.email = other.email _ = self.years.merge(other.years) return self @property def person(self) -> str: result = self.name if self.name and self.email: result += " " if self.email: result += f"<{self.email}>" return result @t.override def __str__(self) -> str: result = str(self.years) result += ", " if result else "" result += self.person return result @t.override def __repr__(self) -> str: return str(self) def as_tuple(self) -> tuple[str, str, str]: """Return a tuple representation of the CopyrightHolder.""" years_str = str(self.years) if self.years else "" return (self.name, self.email, years_str) @dataclasses.dataclass(frozen=True) class Options: """Options dataclass for decopy configuration. Frozen: the run's settings are resolved once (decopy._resolve_run_options settles the git-changed scope and Files-Excluded patterns into a fresh Options) and then read as an invariant by every downstream consumer. Build a modified copy with dataclasses.replace rather than mutating in place. """ mode: t.Literal["full", "partial", "changed"] copyright_file: str debug: bool text: bool group_by: t.Literal["license", "copyright"] split_on_license: bool split_debian: bool glob: bool progress: bool jobs: int output: str root: str files: list[str] exclude_fullname_re: rex.Pattern exclude_special_re: rex.Pattern exclude_file_re: rex.Pattern exclude_directory_re: rex.Pattern spdx: bool = False explain: bool = False since: str = "" expand_archives: bool = True max_archive_depth: int = 1 max_archive_member_size: int = 10 * 1024 * 1024 use_gitignore: bool = True files_excluded: tuple[str, ...] = () fill_licenses: bool = True @property def scoped_to_subset(self) -> bool: """Whether the run is limited to a file subset rather than the whole tree. Partial and changed modes both restrict processing and removal to the included files; only full mode walks and reports over everything. """ return self.mode in ("partial", "changed") decopy-0.3.2/decopy/common/log.py000066400000000000000000000015621522176660400167220ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2025, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. import logging logger: logging.Logger = logging.getLogger("decopy") decopy-0.3.2/decopy/common/re_types.py000077500000000000000000000041241522176660400177730ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. from __future__ import annotations import collections.abc as abc import re as reo import typing as t from decopy.common._regex import rex from decopy.common.core_types import CopyrightHolder class ReSub(t.NamedTuple): pattern: rex.Pattern[str] | reo.Pattern[str] repl: str class ReSubGroup(t.NamedTuple): """Ordered group of ReSub rules; only the first matching rule fires.""" subs: tuple[ReSub, ...] class ReFlags(t.NamedTuple): re: str flags: rex.RegexFlag | reo.RegexFlag | int = rex.IGNORECASE type ParseResult = tuple[list[CopyrightHolder], list[str]] class ReLicense(t.NamedTuple): re: str license: str # get_detail returns the refined licence id, or None to suppress this whole # family (used when an aggregate/collection should be reported under one id # detected by another family). get_detail: ( abc.Callable[[str, rex.Match[str] | reo.Match[str], str], str | None] | None ) = None re_flags: rex.RegexFlag | reo.RegexFlag = rex.IGNORECASE # Exception SUBS applied to a matched licence id (the " with X exception" # suffix). When None, find_licenses falls back to EXCEPTION_SUBS_MAP keyed by # family; carrying it on the row keeps the rule self-contained. exception_subs: tuple[ReSub | ReSubGroup, ...] | None = None decopy-0.3.2/decopy/decopy.py000077500000000000000000000240021522176660400161310ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. from __future__ import annotations import dataclasses from decopy.cmdoptions import process_options from decopy.common.core_types import Options from decopy.common.log import logger from decopy.dep5doc.copyright_processor import resolve_copyright_path from decopy.dep5doc.copyright_reader import read_copyright, read_files_excluded from decopy.dep5doc.delta import ( NEW_PARAGRAPH, ChangedDelta, DeltaEntry, DeltaReport, NullDelta, PartialDelta, freeze_holder_sources, ) from decopy.dep5doc.dep5 import Copyright, Group from decopy.dep5doc.output import generate_explain, generate_output from decopy.dep5doc.renderer import renderer_for from decopy.fstree import vcs from decopy.fstree.nodes import FileInfo from decopy.fstree.tree import RootInfo from decopy.grouping.file_policy import group_key, holder_sources, license_source from decopy.grouping.group_contracts import GroupProtocol from decopy.grouping.grouping import GroupKey def _assigned_files(copyright_: Copyright) -> set[FileInfo]: """Files still held by an existing group after remove_misplaced_files. Those files matched their existing paragraph, so the assembly loop must not place them in a fresh group a second time. """ return {fileinfo for group in copyright_.groups or () for fileinfo in group.files} def prepare_output_groups( filetree: RootInfo, copyright_: Copyright, options: Options, ) -> dict[GroupKey, GroupProtocol]: assigned: set[FileInfo] = _assigned_files(copyright_) groups: dict[GroupKey, GroupProtocol] = copyright_.get_group_dict(options) for fileinfo in filetree.walk_files(): if options.scoped_to_subset and not fileinfo.included: continue # if already assigned, the file matched its copyright group if fileinfo in assigned: continue # The file is not covered by any kept group, so it gets a fresh one. file_key = group_key(fileinfo, options) group = groups.setdefault(file_key, Group(file_key)) group.add_file(fileinfo) assigned.add(fileinfo) return groups def _copyright_relpath(options: Options) -> str: """The copyright file path relative to the tree root, matching fullname. Uses the shared resolver so the in-tree debian/copyright is identified the same way the rest of the pipeline refers to it. Keyed off options.root (== RootInfo.root) so it can run before the tree is built. """ filepath = resolve_copyright_path(options) try: return str(filepath.relative_to(options.root)) except ValueError: return str(filepath) def _resolve_run_options(options: Options) -> tuple[Options, tuple[str, ...]]: """Settle the run's scope into a fresh Options plus the deleted-paths set. Loads the header Files-Excluded patterns and, in changed mode, unions the git-changed set into the file scope. Returns a *new* Options (the input is never mutated, so every downstream consumer reads an invariant) and the paths removed since the baseline (deletions plus the old side of renames), which name the deletions that may have emptied a paragraph (stale-paragraph provenance). On any git unavailability the run degrades to a full scan (mode reset to "full") after one warning, with () deleted. """ files_excluded = read_files_excluded(options) if options.mode != "changed": return dataclasses.replace(options, files_excluded=files_excluded), () scope = vcs.changed_paths(options.root, _copyright_relpath(options), options.since) if isinstance(scope, vcs.Fallback): logger.warning("%s", scope.message) return ( dataclasses.replace(options, mode="full", files_excluded=files_excluded), (), ) renamed_new = {new for _, new in scope.renamed} files = sorted(set(options.files) | set(scope.added_or_modified) | renamed_new) deleted = tuple(sorted(set(scope.deleted) | {old for old, _ in scope.renamed})) resolved = dataclasses.replace(options, files=files, files_excluded=files_excluded) return resolved, deleted def _skip_new_file( fileinfo: FileInfo, options: Options, copyright_relpath: str ) -> bool: """A file does not seed a new paragraph when partial mode excludes it or it is the in-tree copyright file itself (its presence is not a staleness signal). Whether it is already filed is decided by the assigned set in _new_paragraph_buckets, so a relicensed file (removed from its group) is no longer hidden here.""" if options.scoped_to_subset and not fileinfo.included: return True return fileinfo.fullname == copyright_relpath def _new_paragraph_buckets( filetree: RootInfo, copyright_: Copyright, options: Options ) -> dict[GroupKey, list[FileInfo]]: """Bucket the files regeneration would (re)group into fresh paragraphs. A file seeds a new paragraph when it is not kept in an existing paragraph (uncovered, or relicensed and removed from its group) and its destination key is not an existing paragraph's key (else it joins that paragraph). Both tests are needed: the kept set catches a file a no-copyright fallback rescued; the keys catch a move into a paragraph that already declares the destination license. """ assigned = _assigned_files(copyright_) existing = {group.get_key(options) for group in copyright_.groups or ()} copyright_relpath = _copyright_relpath(options) buckets: dict[GroupKey, list[FileInfo]] = {} for fileinfo in filetree.walk_files(): if fileinfo in assigned or _skip_new_file(fileinfo, options, copyright_relpath): continue key = group_key(fileinfo, options) if key in existing: continue buckets.setdefault(key, []).append(fileinfo) return buckets def _record_new_paragraphs( buckets: dict[GroupKey, list[FileInfo]], delta: DeltaReport ) -> None: """Record each bucket of (re)grouped files as the paragraph regeneration would add: its files, the license and where it was found, and each holder's source files. The NullDelta sink absorbs the call on generation runs. """ for key, bucket in buckets.items(): files = sorted(bucket, key=lambda fileinfo: fileinfo.fullname) delta.add( DeltaEntry( kind=NEW_PARAGRAPH, detail=key.license, now=key.license, files=tuple(fileinfo.fullname for fileinfo in files), license_source=license_source(files[0]), holder_sources=freeze_holder_sources(holder_sources(files)), ) ) def _delta_sink(options: Options) -> DeltaReport: """Pick the recording sink for this run. Generation runs get the no-op NullDelta (byte-for-byte unchanged). Explain runs collect into a DeltaReport, a ChangedDelta (keeps the whole-tree-sound stale-* but drops drift) in changed mode, or a PartialDelta that also drops stale-* when scoped to a hand-named subset. The changed branch is the one place changed mode diverges from partial. """ if not options.explain: return NullDelta() if options.mode == "changed": return ChangedDelta() if options.scoped_to_subset: return PartialDelta() return DeltaReport() def run_pipeline(options: Options) -> int: """Run the full options -> tree -> dep5 -> output flow for `options`. Shared by main() and the benchmark/golden-output tooling so they all exercise the exact same code path. Returns the process exit code: 0 for a normal run, and in explain mode 1 when the report has entries else 0. """ # Resolve the git-changed scope first, into a fresh (still immutable) # Options: on a fallback it resets the mode to "full", which the sink # selection below must see so a degraded run gets a full DeltaReport rather # than a PartialDelta. Rebinding `options` here means every step below reads # the settled scope. options, deleted = _resolve_run_options(options) report: DeltaReport = _delta_sink(options) filetree: RootInfo = RootInfo.build(options) copyright_ = read_copyright(options) copyright_.process(filetree, report, deleted) filetree.process(options) copyright_.remove_misplaced_files(options, None, report) groups = prepare_output_groups(filetree, copyright_, options) renderer = renderer_for(options) if options.explain: # New-paragraph recording is explain-only; skipping it on generation # runs avoids a second full tree walk (and group_key recompute) whose # deltas the NullDelta sink would only discard. buckets = _new_paragraph_buckets(filetree, copyright_, options) _record_new_paragraphs(buckets, report) generate_explain(report, options) # The report owns stdout; with -o (a real file, not the stdout # sentinel) also regenerate the fresh copyright to that file. The # in-tree debian/copyright is never touched. if options.output: generate_output(groups, filetree, copyright_, options, renderer) return 1 if report.has_staleness() else 0 generate_output(groups, filetree, copyright_, options, renderer) return 0 def main() -> int: options: Options = process_options() return run_pipeline(options) decopy-0.3.2/decopy/dep5doc/000077500000000000000000000000001522176660400156165ustar00rootroot00000000000000decopy-0.3.2/decopy/dep5doc/__init__.py000066400000000000000000000000001522176660400177150ustar00rootroot00000000000000decopy-0.3.2/decopy/dep5doc/copyright_contracts.py000066400000000000000000000043461522176660400222670ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """ Copyright interface and data contracts extracted from dep5.py This module defines the interfaces and data structures that Copyright classes must implement, enabling better separation of concerns and testing. """ from __future__ import annotations import typing as t import debian.copyright as dc from decopy.common.core_types import LicenseRegistry, Options from decopy.fstree.nodes import FileInfo from decopy.fstree.tree import RootInfo from decopy.grouping.group_contracts import GroupProtocol @t.runtime_checkable class CopyrightProtocol(t.Protocol): """ Protocol defining the interface that all Copyright classes must implement. This protocol allows for different Copyright implementations while maintaining a consistent interface. """ groups: list[GroupProtocol] | None registry: LicenseRegistry @property def header(self) -> dc.Header: """The document header (read-only; consumers only read it).""" ... def process(self, filetree: RootInfo) -> None: """Process the filetree to match files to copyright information.""" ... def remove_misplaced_files( self, options: Options, assigned: set[FileInfo] | None = None ) -> None: """Remove files from groups where they no longer belong.""" ... def get_group_dict(self, options: Options) -> dict[t.Any, t.Any]: """Get a dictionary of groups keyed by their group keys.""" ... decopy-0.3.2/decopy/dep5doc/copyright_processor.py000066400000000000000000000325201522176660400223010ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Copyright processing and management""" from __future__ import annotations import fnmatch import typing as t from pathlib import Path import debian.copyright as dc from decopy.common.core_types import LicenseRegistry, Options from decopy.common.log import logger from decopy.dep5doc import staleness from decopy.dep5doc.copyright_contracts import CopyrightProtocol from decopy.dep5doc.delta import DeltaReport, NullDelta from decopy.fstree.nodes import FileInfo from decopy.fstree.tree import RootInfo from decopy.grouping import file_policy from decopy.grouping.group import Group from decopy.grouping.group_contracts import GroupProtocol from decopy.grouping.grouping import GroupKey from decopy.grouping.grouping_strategy import grouping_strategy_for from decopy.licensing.holders import get_copyright_holders from decopy.licensing.license_expr import Node, license_atoms from decopy.licensing.spdx import canonical_license_name, parse_license_synopsis type FilesGroups = list[tuple[dc.FilesParagraph, set[str]]] # Shared no-op sink used as the default when no real report is threaded in. # NullDelta.add() stores nothing, so a single instance is safe to share and # lets the recording sites call delta.add() without a None guard. _NULL_DELTA = NullDelta() def resolve_copyright_path(options: Options) -> Path: """Absolute path to the copyright file, honouring an absolute setting. Relative paths are resolved against ``options.root`` (== ``RootInfo.root``), so it can run before the tree is built. The single source of truth for where the in-tree debian/copyright lives, shared by every consumer that reads or refers to it. """ filepath = Path(options.copyright_file) if filepath.is_absolute(): return filepath return Path(options.root) / filepath def _parse_synopsis(synopsis: str) -> tuple[list[str], Node | None]: """Parse a Files-paragraph License synopsis into (atom keys, expression). Returns the licence atoms to key the group by and the parsed expression to keep for re-emission. An empty or malformed synopsis (which DEP-5 cannot represent) degrades to a single literal key and no expression. """ try: expr = parse_license_synopsis(synopsis) except ValueError: return [synopsis], None return license_atoms(expr), expr class Copyright(CopyrightProtocol): """Matches a filetree against an existing debian/copyright document. Wraps a debian.copyright.Copyright by *composition* (not inheritance): the parsed document is the boundary to python-debian, and only the pieces decopy needs -- the header and the Files/License paragraphs -- are exposed by delegation. Everything else here (process/remove_misplaced_files/group assembly) is decopy's own. Reading a document off disk lives in copyright_reader; an empty document is the default. """ groups: list[GroupProtocol] | None = None def __init__( self, document: dc.Copyright | None = None, registry: LicenseRegistry | None = None, ) -> None: self._document = document if document is not None else dc.Copyright() self.registry = registry if registry is not None else LicenseRegistry() @property def header(self) -> dc.Header: """The wrapped document's header (a dc.Header).""" return self._document.header def all_files_paragraphs(self) -> t.Iterator[dc.FilesParagraph]: """The wrapped document's Files paragraphs.""" return self._document.all_files_paragraphs() def all_license_paragraphs(self) -> t.Iterator[dc.LicenseParagraph]: """The wrapped document's standalone License paragraphs.""" return self._document.all_license_paragraphs() def process( self, filetree: RootInfo, delta: DeltaReport = _NULL_DELTA, deleted: tuple[str, ...] = (), ) -> None: """Match files in the filetree to copyright information. `deleted` (changed mode) names paths removed since the baseline, used to attribute a now-stale paragraph to the deletion that emptied it. """ self._populate_license_singletons() files_paragraphs: FilesGroups = [] # Scan the files paragraphs in reverse order, that way the files should # match only once. paragraphs = list(self.all_files_paragraphs()) for paragraph in reversed(paragraphs): # apply fnmatch to each file_pattern matched_patterns, files = self._get_matched_patterns(paragraph, filetree) # Nothing in the paragraph matched any files. The paragraph will not # be included in files_paragraphs. if not matched_patterns: logger.info("No matching pattern in:\n%s", paragraph.dump()) staleness.record_stale_paragraph(delta, paragraph, deleted) continue # Some patterns (but not all) failed to match. # Those specific patterns get removed from the paragraph; the dead # ones are recorded here, where both declared and matched sets are # in scope (a paragraph that matched nothing is recorded above as a # single stale paragraph, never as N stale patterns). if len(matched_patterns) != len(paragraph.files): staleness.record_stale_patterns(delta, paragraph, matched_patterns) paragraph.files = matched_patterns # type: ignore[assignment] # ty: ignore[invalid-assignment] files_paragraphs.append((paragraph, files)) self._process_groups(filetree, files_paragraphs) def _populate_license_singletons(self) -> None: """Store each standalone license paragraph in the matching License singleton.""" for license_para in self.all_license_paragraphs(): paragraph_license = t.cast(dc.License, license_para.license) # A standalone stanza header is a single licence name, so canonicalise # it directly (no expression parse) to the same key the Files-paragraph # groups use; otherwise the stored text is keyed by a spelling no group # looks up and is dropped from the output. license_ = self.registry.get( canonical_license_name(paragraph_license.synopsis) ) license_.stored = license_para @staticmethod def _get_matched_patterns( paragraph: dc.FilesParagraph, filetree: RootInfo, ) -> tuple[list[str], set[str]]: """Returns the files that match the patterns for a paragraph.""" matched_patterns: list[str] = [] files: set[str] = set() for file_pattern in paragraph.files: found = False for filename in fnmatch.filter(filetree.names, file_pattern): fileinfo = filetree[filename] # If the file already has a matching pattern, it was already # matched by previous paragraphs. if fileinfo.matching_pattern: continue # Set the matched pattern fileinfo.matching_pattern = file_pattern if not fileinfo.is_container: files.add(filename) if not found: matched_patterns.append(file_pattern) found = True if not found: logger.info("No match found for %s", file_pattern) return matched_patterns, files def _process_groups( self, filetree: RootInfo, files_paragraphs: FilesGroups, ) -> None: """Process files paragraphs and create groups.""" self.groups = [] for position, (paragraph, files) in enumerate(files_paragraphs): group = Group(position=position) group.stored = paragraph para_license = t.cast(dc.License, paragraph.license) para_copyright = t.cast(str, paragraph.copyright) para_comment = t.cast(str, paragraph.comment) # Parse the synopsis (DEP-5 sec 7.2: and/or/comma) into an # expression, keep it for faithful re-emission, and key the group's # licences by its atoms regardless of the and/or shape. A malformed # or empty synopsis degrades to a single literal key. license_keys, group.license_expr = _parse_synopsis(para_license.synopsis) for key in license_keys: group.licenses[key] = self.registry.get(key) # Does this file paragraph includes the license text? # Can we reuse it? if len(license_keys) == 1 and para_license.text: license_ = self.registry.get(license_keys[0]) if not license_.stored: license_.stored = dc.LicenseParagraph.create(para_license) holders = [] if para_copyright: for value in para_copyright.split("\n"): value_ = value.lstrip() new_holders, _ = get_copyright_holders(value_) holders.extend(new_holders) group.add_copyrights(holders) # Snapshot the declared holders before the file loop (and the later # remove_misplaced_files merge) can fold in any file holders. group.declared_copyright_key = group.copyrights.key() for filename in files: fileinfo = filetree[filename] fileinfo.existing_group = group group.add_file(fileinfo) if para_comment: group.comments.append(para_comment) self.groups.append(group) def remove_misplaced_files( self, options: Options, assigned: set[FileInfo] | None = None, delta: DeltaReport = _NULL_DELTA, ) -> None: """Remove files from groups where they no longer belong. This method should be called after files have been processed and their licensing and copyright information has been obtained. Files that do not match their placement in the copyright file get removed from those groups. No files are added to existing or new groups. Files that keep their placement are added to `assigned` (a caller-owned set), so a later assembly pass knows they are already grouped. When the caller does not need that, `assigned` is left None and tracked locally. """ if assigned is None: assigned = set() strategy = grouping_strategy_for(options.group_by) for group in self.groups or (): group_key = group.get_key(options) group_key_no_copyright = group.get_key(options, ignore_copyright=True) logger.debug("Keys: %s %s", group_key, group_key_no_copyright) # Make an explicit copy to be able to remove items from the list. for fileinfo in list(group.files): # In a scoped run, only included files are taken into account if options.scoped_to_subset and not fileinfo.included: group.files.del_file(fileinfo) continue # Check if the current key for the file matches the group key. file_key = file_policy.group_key(fileinfo, options) logger.debug("File: %s %s", fileinfo, file_key) remove = not strategy.keeps_misplaced( file_key, group_key, group_key_no_copyright ) # Either delete from the group or record the file as assigned, # to show that the file has matched its group. if remove: staleness.record_relicensed(delta, fileinfo, (file_key, group_key)) group.files.del_file(fileinfo) else: # Update parsed copyrights group.copyrights.merge(fileinfo.copyrights) assigned.add(fileinfo) # The kept files are settled; report any drift between them and the # paragraph's declared Copyright field. staleness.record_copyright_drift(delta, group, strategy) def get_group_dict(self, options: Options) -> dict[GroupKey, GroupProtocol]: """Return a dictionary of groups, keyed by their key.""" groups: dict[GroupKey, GroupProtocol] = {} for group in self.groups or (): key = group.get_key(options) if key in groups: # If the group already exists, add the files to it for fileinfo in group.files: groups[key].add_file(fileinfo) else: groups[key] = group return groups decopy-0.3.2/decopy/dep5doc/copyright_reader.py000066400000000000000000000060601522176660400215240ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Reading an existing debian/copyright file (the persistence seam). This module owns the file I/O that turns a path into a parsed Copyright (or the header's Files-Excluded patterns). Keeping it separate from copyright_processor leaves the Copyright class to its one job -- matching files to groups -- rather than also reading files off disk. The dependency is one-way: this module imports Copyright (and the shared path resolver), never the reverse. """ from __future__ import annotations import debian.copyright as dc from decopy.common.core_types import LicenseRegistry, Options from decopy.dep5doc.copyright_processor import Copyright, resolve_copyright_path def read_copyright( options: Options, registry: LicenseRegistry | None = None ) -> Copyright: """Read the existing copyright file into a Copyright, or an empty one. Returns an empty Copyright when the file is absent or not machine-readable. """ filepath = resolve_copyright_path(options) if not filepath.exists(): return Copyright(registry=registry) try: with filepath.open() as c: return Copyright(dc.Copyright(c, strict=False), registry=registry) except (dc.NotMachineReadableError, ValueError): return Copyright(registry=registry) def _read_header(options: Options) -> dc.Header | None: """The existing copyright file's header, or None if absent/unparseable. Read before the tree walk so the Files-Excluded patterns are available to prune the walk. """ filepath = resolve_copyright_path(options) if not filepath.exists(): return None try: with filepath.open() as c: return dc.Copyright(c, strict=False).header except (dc.NotMachineReadableError, ValueError): return None def read_files_excluded(options: Options) -> tuple[str, ...]: """The header Files-Excluded patterns, whitespace-split, or () if absent. DEP-5 lists the repack patterns whitespace-separated (possibly spanning continuation lines); python-debian returns the raw lines, so each is split into individual patterns. """ header = _read_header(options) if header is None: return () return tuple( pattern for line in (header.files_excluded or ()) for pattern in line.split() ) decopy-0.3.2/decopy/dep5doc/delta.py000066400000000000000000000153551522176660400172720ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Delta data types for the explain mode. A DeltaReport collects the semantic staleness decopy detects while it regenerates debian/copyright (stale paragraphs, stale patterns, relicensed files and new files), so the explain renderer can report them instead of discarding them. NullDelta is the no-op sink used on normal generation runs, keeping the recording sites branch-free. """ from __future__ import annotations import dataclasses import typing as t if t.TYPE_CHECKING: from decopy.grouping.grouping import GroupKey # Kind constants STALE_PARAGRAPH = "stale-paragraph" STALE_PATTERN = "stale-pattern" RELICENSED = "relicensed" NEW_FILE = "new-file" NEW_PARAGRAPH = "new-paragraph" COPYRIGHT_DRIFT = "copyright-drift" # Relicensed sub-kinds SUB_LICENSE = "license-changed" SUB_COPYRIGHT = "copyright-changed" SUB_BOTH = "license-and-copyright-changed" # Copyright-drift sub-kinds. A removed-only drift is advisory (regeneration # unions holders and never drops one), so it reports but does not fail the gate. DRIFT_ADDED = "holders-added" DRIFT_REMOVED = "holders-removed" DRIFT_BOTH = "holders-added-and-removed" @dataclasses.dataclass(frozen=True) class DeltaEntry: """A single recorded staleness item. Only the slots relevant to the kind are populated; the rest stay None. """ kind: str detail: str paragraph: str | None = None pattern: str | None = None path: str | None = None was: str | None = None now: str | None = None sub_kind: str | None = None was_copyright: tuple[str, ...] | None = None now_copyright: tuple[str, ...] | None = None paragraph_license: str | None = None # Provenance for explain mode. files lists a NEW_PARAGRAPH bucket; # license_source phrases where the license was found; holder_sources maps # each holder key to the files it was detected in. Tuples keep the frozen # dataclass hashable. files: tuple[str, ...] | None = None license_source: str | None = None holder_sources: tuple[tuple[str, tuple[str, ...]], ...] | None = None # Changed mode: the deleted paths that would have matched a now-stale # paragraph's patterns (the deletions that emptied it); None when nothing is # attributable or outside changed mode. deleted_files: tuple[str, ...] | None = None def freeze_holder_sources( sources: dict[str, tuple[str, ...]], ) -> tuple[tuple[str, tuple[str, ...]], ...]: """A holder -> source-files map in the frozen tuple-of-pairs form a DeltaEntry carries (sorted by holder for stable output and equality).""" return tuple(sorted(sources.items())) def is_advisory(entry: DeltaEntry) -> bool: """Whether an entry reports without counting as staleness. Only a removed-only copyright drift is advisory: it lists a holder no file backs, which regeneration would not drop, so it is information, not a gate failure. """ return entry.kind == COPYRIGHT_DRIFT and entry.sub_kind == DRIFT_REMOVED class DeltaReport: """Ordered collection of DeltaEntry items grouped on demand by kind.""" def __init__(self) -> None: self._entries: list[DeltaEntry] = [] def add(self, entry: DeltaEntry) -> None: self._entries.append(entry) def is_empty(self) -> bool: return not self._entries def has_staleness(self) -> bool: """Whether any recorded entry is a staleness signal (drives the exit code). Advisory entries (a removed-only copyright drift) are reported but do not count, so a tree clean except for advice still exits 0. """ return any(not is_advisory(entry) for entry in self._entries) def by_kind(self) -> dict[str, list[DeltaEntry]]: grouped: dict[str, list[DeltaEntry]] = {} for entry in self._entries: grouped.setdefault(entry.kind, []).append(entry) return grouped class NullDelta(DeltaReport): """No-op sink. add() does nothing; is_empty() is always True.""" def add(self, entry: DeltaEntry) -> None: pass class PartialDelta(DeltaReport): """Explain sink for partial mode: drops stale-* and copyright-drift. Only new-file, new-paragraph and relicensed entries are sound for a requested subset (all concern included files). The stale-* kinds need the whole tree, and so does copyright drift (a removed holder may live on a file outside the subset), so all three are silently ignored, keeping the recording sites branch-free. """ def add(self, entry: DeltaEntry) -> None: if entry.kind in (STALE_PARAGRAPH, STALE_PATTERN, COPYRIGHT_DRIFT): return super().add(entry) class ChangedDelta(DeltaReport): """Explain sink for changed mode: keeps stale-* but drops copyright-drift. Unlike a plain subset (PartialDelta), changed mode keeps stale-paragraph and stale-pattern: they are whole-tree-sound because Copyright.process matches each paragraph's patterns against the complete filetree before any scoping, so a paragraph matching nothing is real regardless of what was parsed. Drift stays dropped for the same reason PartialDelta drops it: the kept-file holder union only reflects the included files under scoped parsing, so a drift there would be a false positive. """ def add(self, entry: DeltaEntry) -> None: if entry.kind == COPYRIGHT_DRIFT: return super().add(entry) def relicense_sub_kind(file_key: GroupKey, group_key: GroupKey) -> str | None: """Classify a relicense by comparing the strategy-masked group keys. Only the license (slot 0) and copyright (slot 1) slots are compared; the path (slot 2) is not a relicense signal. Returns None when neither differs. """ license_changed = file_key.license != group_key.license copyright_changed = file_key.copyright != group_key.copyright if license_changed and copyright_changed: return SUB_BOTH if license_changed: return SUB_LICENSE if copyright_changed: return SUB_COPYRIGHT return None decopy-0.3.2/decopy/dep5doc/dep5.py000077500000000000000000000023541522176660400170340ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """DEP-5 copyright handling - re-export façade.""" from __future__ import annotations from decopy.dep5doc.copyright_processor import Copyright as Copyright from decopy.dep5doc.copyright_reader import read_copyright as read_copyright from decopy.dep5doc.copyright_reader import read_files_excluded as read_files_excluded from decopy.grouping.group import Group as Group __all__ = ["Copyright", "Group", "read_copyright", "read_files_excluded"] decopy-0.3.2/decopy/dep5doc/explain_renderer.py000066400000000000000000000265261522176660400215310ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Explain-mode renderer: a DeltaReport rendered as staleness report text. This is a separate output format from the DEP-5 document Renderer (renderer.py): it does not implement the `Renderer` protocol and changes for its own reasons, so it lives in its own module. output.generate_explain drives it. """ from __future__ import annotations import typing as t from decopy.dep5doc.delta import ( COPYRIGHT_DRIFT, NEW_PARAGRAPH, RELICENSED, STALE_PARAGRAPH, STALE_PATTERN, ) from decopy.licensing.spdx import render_license_name if t.TYPE_CHECKING: from decopy.common.core_types import Options from decopy.dep5doc.delta import DeltaEntry, DeltaReport def _pluralise(count: int, singular: str, plural: str) -> str: return singular if count == 1 else plural def _change_phrase(was: str, now: str) -> str: """The shared `was: X -> now: Y` license-transition phrasing.""" return f"was: {was} -> now: {now}" def _item_line(path: str, width: int, phrase: str) -> str: """A 4-space-indented item: the path left-padded to `width`, then the phrase. Padding to the section's widest path aligns the `was:` column across rows. """ return f" {path.ljust(width)} {phrase}" # A removed-holder copyright drift is advisory (regeneration unions holders and # never drops one), so it reports but does not fail the gate. The terse tag marks # the line; a footnote under the section header explains it once. _ADVISORY_TAG = "[advisory]" _DRIFT_FOOTNOTE = ( f" ({_ADVISORY_TAG} holders are kept on regeneration; does not fail the gate)" ) # Cap the per-holder source list so a holder in many files stays terminal # friendly; the overflow collapses to "+N more". _SOURCE_CAP = 3 def _source_suffix(files: tuple[str, ...] | None) -> str: """` (in a, b)` provenance for an added holder, truncated past the cap.""" if not files: return "" shown = files[:_SOURCE_CAP] listed = ", ".join(shown) extra = len(files) - len(shown) if extra: listed += f" +{extra} more" return f" (in {listed})" def _holder_delta_lines(entry: DeltaEntry, removed_suffix: str = "") -> list[str]: """Indented `+ added` / `- removed` holder lines, sorted within each side. Shared by the copyright-drift block and the relicensed section (copyright grouping) so both render holder changes identically. Under license grouping the relicensed copyright slots are None, so this yields no lines. An added holder names the files it was found in (entry.holder_sources); a removed holder is absent from every file, so it carries no source. `removed_suffix` lets the drift block tag its removed lines as advisory. """ was_set = set(entry.was_copyright or ()) now_set = set(entry.now_copyright or ()) sources = dict(entry.holder_sources or ()) lines = [ f" + {holder}{_source_suffix(sources.get(holder))}" for holder in sorted(now_set - was_set) ] lines += [ f" - {holder}{removed_suffix}" for holder in sorted(was_set - now_set) ] return lines # A new-paragraph block aligns its field values past the widest label, so Files # continuation lines sit under the first file. _FIELD_WIDTH = len("Copyright:") def _field_lines(label: str, values: list[str]) -> list[str]: """A `Label: value` field, value-aligned, with extra values continued below.""" head = f" {label.ljust(_FIELD_WIDTH)} {values[0]}" pad = " " + " " * _FIELD_WIDTH + " " return [head, *(f"{pad}{value}" for value in values[1:])] def _holder_field_values( holder_sources: tuple[tuple[str, tuple[str, ...]], ...] | None, ) -> list[str]: """Each holder with its `(in )` source, sorted, for a Copyright field.""" return [ f"{holder}{_source_suffix(files)}" for holder, files in sorted(dict(holder_sources or ()).items()) ] class ExplainRenderer: """Renders a DeltaReport as the explain-mode staleness report text. The output owns a fixed section order (new files, relicensed, stale paragraphs, stale patterns); empty sections are skipped. Every license is passed through render_license_name so --spdx is honoured, and relicensed holders are shown only under --group-by copyright. """ CLEAN_MESSAGE = "debian/copyright is up to date." LEAD_IN = "debian/copyright is out of date:" def explain(self, report: DeltaReport, options: Options) -> str: if report.is_empty(): return f"{self.CLEAN_MESSAGE}\n" grouped = report.by_kind() sections = [ self._relicensed_section(grouped.get(RELICENSED, []), options), self._copyright_drift_section(grouped.get(COPYRIGHT_DRIFT, []), options), self._new_paragraph_section(grouped.get(NEW_PARAGRAPH, []), options), self._stale_paragraph_section(grouped.get(STALE_PARAGRAPH, []), options), self._stale_pattern_section(grouped.get(STALE_PATTERN, []), options), ] blocks = [self.LEAD_IN, *(s for s in sections if s)] return "\n\n".join(blocks) + "\n" def _new_paragraph_section( self, entries: list[DeltaEntry], options: Options ) -> str: entries = [e for e in entries if e.files] if not entries: return "" entries = sorted(entries, key=lambda e: (e.now or "", e.files or ())) count = len(entries) header = " {} {} would be added:".format( count, _pluralise(count, "new paragraph", "new paragraphs") ) blocks = [self._new_paragraph_block(e, options) for e in entries] return "\n\n".join([header, *blocks]) @classmethod def _new_paragraph_block(cls, entry: DeltaEntry, options: Options) -> str: """One would-be Files paragraph: its files, License (with source) and, when any holder was detected, a Copyright field with per-holder sources. """ license_ = render_license_name(entry.now or "", options) source = entry.license_source or "" lines = _field_lines("Files:", list(entry.files or ())) lines += _field_lines("License:", [f"{license_} ({source})"]) holders = _holder_field_values(entry.holder_sources) if holders: lines += _field_lines("Copyright:", holders) return "\n".join(lines) def _relicensed_section(self, entries: list[DeltaEntry], options: Options) -> str: if not entries: return "" count = len(entries) header = " {} {}:".format( count, _pluralise( count, "file no longer matches its paragraph", "files no longer match their paragraph", ), ) width = max(len(e.path or "") for e in entries) blocks = [self._relicensed_block(e, options, width) for e in entries] return "\n".join([header, *blocks]) @staticmethod def _relicensed_block(entry: DeltaEntry, options: Options, width: int) -> str: """The relicensed item line plus any holder-delta lines below it.""" was = render_license_name(entry.was or "", options) now = render_license_name(entry.now or "", options) line = _item_line(entry.path or "", width, _change_phrase(was, now)) holders = _holder_delta_lines(entry) return "\n".join([line, *holders]) def _copyright_drift_section( self, entries: list[DeltaEntry], options: Options ) -> str: _ = options if not entries: return "" count = len(entries) header = " {} {}:".format( count, _pluralise( count, "paragraph has copyright drift", "paragraphs have copyright drift", ), ) blocks = [self._drift_block(e) for e in entries] # Explain the terse [advisory] tag once, only when one actually appears. if any(_ADVISORY_TAG in block for block in blocks): return "\n".join([header, _DRIFT_FOOTNOTE, *blocks]) return "\n".join([header, *blocks]) @staticmethod def _drift_block(entry: DeltaEntry) -> str: """The Files line plus the +added / -removed (advisory) holder lines.""" lines = [ f" {line}" for line in (entry.paragraph or "").splitlines() if line.startswith("Files:") ] lines += _holder_delta_lines(entry, removed_suffix=f" {_ADVISORY_TAG}") return "\n".join(lines) def _stale_paragraph_section( self, entries: list[DeltaEntry], options: Options ) -> str: entries = [e for e in entries if e.paragraph] if not entries: return "" count = len(entries) header = " {} {}:".format( count, _pluralise( count, "paragraph matches no files", "paragraphs match no files", ), ) blocks = [self._stale_paragraph_block(e, options) for e in entries] return "\n".join([header, *blocks]) @staticmethod def _stale_paragraph_block(entry: DeltaEntry, options: Options) -> str: """The paragraph's Files line(s) with the license as a tag on the last. The separate License: line is dropped in favour of a `[License: X]` tag, matching the stale-pattern line so the two dead-container views read alike. A deleted file that would have matched the paragraph is named beneath it, one per line, as the reason it went stale. """ files_lines = [ f" {line}" for line in (entry.paragraph or "").splitlines() if line.startswith("Files:") ] if files_lines: license_ = render_license_name(entry.paragraph_license or "?", options) files_lines[-1] += f" [License: {license_}]" culprit_lines = [ f" {path} (deleted)" for path in (entry.deleted_files or ()) ] return "\n".join([*files_lines, *culprit_lines]) def _stale_pattern_section( self, entries: list[DeltaEntry], options: Options ) -> str: if not entries: return "" count = len(entries) header = " {} {}:".format( count, _pluralise(count, "stale pattern", "stale patterns") ) lines = [self._stale_pattern_line(e, options) for e in entries] return "\n".join([header, *lines]) @staticmethod def _stale_pattern_line(entry: DeltaEntry, options: Options) -> str: license_ = render_license_name(entry.paragraph_license or "?", options) return f" {entry.pattern} matches nothing [License: {license_}]" decopy-0.3.2/decopy/dep5doc/output.py000077500000000000000000000107211522176660400175340ustar00rootroot00000000000000#!/usr/bin/env python3 # # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. from __future__ import annotations import sys import typing as t from collections.abc import Generator, Mapping from contextlib import contextmanager from pathlib import Path from typing import IO from decopy.common.core_types import LicenseRegistry, Options from decopy.common.log import logger from decopy.dep5doc.explain_renderer import ExplainRenderer from decopy.dep5doc.renderer import Renderer from decopy.fstree.tree import RootInfo from decopy.grouping.group import TallyTracker from decopy.grouping.group_contracts import GroupProtocol if t.TYPE_CHECKING: from decopy.dep5doc.copyright_contracts import CopyrightProtocol from decopy.dep5doc.delta import DeltaReport @contextmanager def open_output_file(filename: str) -> Generator[IO[str], None, None]: if filename: with Path(filename).open("wt", encoding="utf-8") as f: yield f else: yield sys.stdout def _tally_partial_parents( filetree: RootInfo, options: Options, tracker: TallyTracker ) -> None: """In a scoped run, mark the requested files' parents as tallied. This avoids emitting a 'Files: *' paragraph that would claim the whole tree. """ if not options.scoped_to_subset: return for item in options.files: parent = filetree[item].parent if not parent: continue tracker.tally(parent) def generate_explain(report: DeltaReport, options: Options) -> None: """Render the staleness report to stdout (the report destination is fixed).""" print(ExplainRenderer().explain(report, options), end="") def generate_output( groups: Mapping[t.Any, GroupProtocol], filetree: RootInfo, copyright_: CopyrightProtocol | None, options: Options, renderer: Renderer, ) -> None: # The Copyright owns the license registry it populated while reading the # existing file; reuse it so stored license texts reach the stanzas. registry = copyright_.registry if copyright_ is not None else LicenseRegistry() # Track the licenses that are in use licenses: set[str] = set() # Shared across all groups so a later, more specific group does not # re-wildcard an ancestor an earlier group already covered. tracker = TallyTracker() def log_generated(label: str, text: str) -> None: # Only when writing to a real file: on stdout the text is already # visible, so echoing it into the log stream would just duplicate it. if options.output: logger.debug("Generated %s:\n%s", label, text) with open_output_file(options.output) as f: # Print header header = renderer.header(copyright_, options) print(header, file=f) log_generated("header", header) _tally_partial_parents(filetree, options, tracker) # Print files paragraphs for _, group in sorted(groups.items(), key=lambda i: i[1].sort_key(options)): if not group.copyright_block_valid(): continue licenses.update(group.licenses.keys()) if options.glob: file_patterns = group.files.get_patterns(tracker) else: # When called with --no-glob, only list the file names file_patterns = group.files.sorted_members() paragraph = renderer.file_block(group, options, file_patterns) print(file=f) print(paragraph, file=f) log_generated("group", paragraph) # Print license paragraphs for key in sorted(licenses): paragraph = renderer.license_block(registry.get(key), options) print(file=f) print(paragraph, file=f) log_generated("license block", paragraph) decopy-0.3.2/decopy/dep5doc/renderer.py000066400000000000000000000202551522176660400200020ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Output format as an injectable Renderer. The document format is an axis decopy is expected to vary (DEP-5 today, perhaps an SPDX document or a JSON summary later). This module gathers the pieces of a rendered document behind a single Renderer so the orchestration in output.py can depend on the abstraction rather than hard-coding DEP-5. A Renderer produces three pieces; the orchestration owns the surrounding plumbing (writing, blank-line separators, choosing which groups/licenses to emit): * header(copyright_, options) - the leading Format/Source paragraph * file_block(group, options, file_patterns) - one Files paragraph for a group * license_block(license_, options) - one standalone License stanza Dep5Renderer is the default implementation (DEP-5 copyright-format 1.0); output.py selects a renderer via renderer_for() and drives it. """ from __future__ import annotations import copy import typing as t from decopy.common.core_types import License from decopy.licensing.license_expr import render_spdx_expression from decopy.licensing.license_texts import LicenseTextResolver from decopy.licensing.spdx import dep5_to_spdx, exception_to_spdx, render_license_name if t.TYPE_CHECKING: from decopy.common.core_types import Options from decopy.dep5doc.copyright_contracts import CopyrightProtocol from decopy.grouping.group_contracts import GroupProtocol @t.runtime_checkable class Renderer(t.Protocol): """Protocol for rendering the pieces of a copyright document.""" def header(self, copyright_: CopyrightProtocol | None, options: Options) -> str: """Render the leading header paragraph.""" ... def file_block( self, group: GroupProtocol, options: Options, file_patterns: list[str] ) -> str: """Render the Files paragraph for a group. `file_patterns` is the already-resolved list of file names/wildcards for the Files: field; the caller owns how it was computed (wildcarding, cross-group tallying) so the renderer holds no shared state. """ ... def license_block(self, license_: License, options: Options) -> str: """Render the standalone License stanza for a license.""" ... class Dep5Renderer(Renderer): """Renders DEP-5 copyright-format 1.0.""" format_url = "https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/" def __init__(self, resolver: LicenseTextResolver | None = None) -> None: """`resolver` fills empty License stanzas; None keeps the TODO placeholder.""" self._resolver = resolver def header(self, copyright_: CopyrightProtocol | None, options: Options) -> str: if options.scoped_to_subset: return "Format: {}\nSource: TODO\nComment: *** only: {} ***".format( self.format_url, ", ".join(options.files), ) if copyright_: # header() is a query: render from a copy so the caller's document # is not mutated (and a second render sees the same input). The copy # keeps the existing field order; Format is canonicalised and a # missing Source defaults to TODO. header = copy.deepcopy(copyright_.header) header.format = self.format_url # type: ignore[assignment] # ty: ignore[invalid-assignment] if not header.source: header.source = "TODO" return (header.dump() or "").rstrip("\n") return f"Format: {self.format_url}\nSource: TODO" @staticmethod def _license_field(group: GroupProtocol, options: Options) -> str: """The rendered License field, honouring --spdx. With --spdx and a parsed expression, render straight from the AST so and/or/comma structure converts faithfully (the string path only knew ' or '). Otherwise emit the DEP-5 field, converted by render_license_name. """ if options.spdx and group.license_expr is not None: return render_spdx_expression( group.license_expr, convert_name=dep5_to_spdx, convert_exception=lambda label: exception_to_spdx(label) or label, ) return render_license_name(group.license_field, options) def file_block( self, group: GroupProtocol, options: Options, file_patterns: list[str] ) -> str: if not group.copyright_block_valid(): return "" files = "\n ".join(file_patterns) block = [f"Files: {files}"] if group.copyrights: holders = "\n ".join(group.copyrights.sorted_members()) block.append(f"Copyright: {holders}") block.append(f"License: {self._license_field(group, options)}") comments = group.get_comments() if comments: block.append(f"Comment: {comments}") # And finally add all the "unknown" fields carried by the stored # paragraph (the group owns which fields count as non-standard). for field, value in group.extra_fields(): block.append(f"{field}: {value}") return "\n".join(block) def license_block(self, license_: License, options: Options) -> str: display = render_license_name(license_.name, options) return self._license_stanza(license_, display) def _license_stanza( self, license_: License, display_name: str | None = None ) -> str: """Render the DEP-5 License stanza, optionally overriding the name. Stored text always wins; otherwise the resolver (when present) fills the body from the licence's DEP-5 name, keyed independently of the displayed name so --spdx does not change the lookup. Without either, the TODO placeholder is kept. """ display = display_name if display_name is not None else license_.name stored = self._stored_stanza(license_, display) if stored is not None: return stored if self._resolver is not None: body = self._resolver.body(license_.name) if body is not None: return self._format_body(display, body) return f"License: {display}\nComment: Add the corresponding license text here" @staticmethod def _stored_stanza(license_: License, display: str) -> str | None: """The stanza rebuilt from stored text, or None when there is none.""" if not (license_.stored and (dump := license_.stored.dump()) is not None): return None body = dump.rstrip("\n") newline_pos = body.find("\n") if newline_pos >= 0: return f"License: {display}{body[newline_pos:]}" return f"License: {display}" @staticmethod def _format_body(display: str, body: str) -> str: """Wrap a plain-text body as a DEP-5 License stanza (space-indented, `.` for blank lines).""" lines = [f"License: {display}"] lines += [f" {line}" if line else " ." for line in body.split("\n")] return "\n".join(lines) def renderer_for(options: Options) -> Renderer: """Select the output renderer for the run. Only DEP-5 copyright-format 1.0 is supported today; this is the single seam where a future --output-format option would choose a different Renderer. The License-stanza resolver is attached here so --no-fill-licenses restores the TODO placeholder. """ resolver = LicenseTextResolver() if options.fill_licenses else None return Dep5Renderer(resolver) decopy-0.3.2/decopy/dep5doc/staleness.py000066400000000000000000000170251522176660400201760ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Explain-mode staleness detection. These functions produce the DeltaEntry records the explain renderer reports: relicensed files, copyright drift, stale paragraphs and stale patterns. They are the *producers* that pair with delta.py's data types, kept out of copyright_processor so that class stays focused on matching files to paragraphs. Each takes the caller's DeltaReport sink and appends to it; on a generation run the sink is the no-op NullDelta, so the calls are unconditional. This module must not import copyright_processor (the dependency runs the other way). """ from __future__ import annotations import fnmatch import typing as t import debian.copyright as dc from decopy.dep5doc.delta import ( COPYRIGHT_DRIFT, DRIFT_ADDED, DRIFT_BOTH, DRIFT_REMOVED, RELICENSED, STALE_PARAGRAPH, STALE_PATTERN, DeltaEntry, DeltaReport, freeze_holder_sources, relicense_sub_kind, ) from decopy.fstree.nodes import FileInfo from decopy.grouping import file_policy from decopy.grouping.group_contracts import GroupProtocol from decopy.grouping.grouping import GroupKey from decopy.grouping.grouping_strategy import GroupingStrategy from decopy.licensing.holders import CopyrightHolders def record_relicensed( delta: DeltaReport, fileinfo: FileInfo, keys: tuple[GroupKey, GroupKey], ) -> None: """Record a relicensed file when its key no longer matches its group key. No entry is recorded for a path-only difference (relicense_sub_kind None); the NullDelta sink absorbs the call on generation runs. Packaging files under debian/ are skipped: debian/copyright is their source of truth, so a detected license mismatch there is not a staleness signal. """ if fileinfo.is_packaging(): return file_key, group_key = keys sub_kind = relicense_sub_kind(file_key, group_key) if sub_kind is None: return delta.add( DeltaEntry( kind=RELICENSED, detail=fileinfo.fullname, path=fileinfo.fullname, was=group_key.license, now=file_key.license, sub_kind=sub_kind, was_copyright=group_key.copyright, now_copyright=file_key.copyright, holder_sources=freeze_holder_sources( file_policy.holder_sources([fileinfo]) ), ) ) def _drift_sub_kind(added: set[str], removed: set[str]) -> str: """Classify a copyright drift by which directions changed.""" if added and removed: return DRIFT_BOTH if added: return DRIFT_ADDED return DRIFT_REMOVED def _upstream_files(group: GroupProtocol) -> list[FileInfo]: """The group's non-packaging kept files. Packaging files (under debian/) are excluded: debian/copyright is their source of truth, so their detected holders must not drive drift or its holder-source provenance. """ return [fileinfo for fileinfo in group.files if not fileinfo.is_packaging()] def _upstream_holder_key(files: list[FileInfo]) -> tuple[str, ...] | None: """Holder key over the given files, or None when there are none (a pure-packaging group, for which drift is not reported at all).""" if not files: return None file_union = CopyrightHolders() for fileinfo in files: file_union.merge(fileinfo.copyrights) return file_union.key() def record_copyright_drift( delta: DeltaReport, group: GroupProtocol, strategy: GroupingStrategy, ) -> None: """Record a kept group whose Copyright field no longer matches its files. Compares the paragraph's declared holders (snapshotted before any merge) against the union of its kept non-packaging files' holders. Only strategies that mask the copyright slot report drift (otherwise the relicensed path already covers it). An added holder is staleness; a removed-only holder is advisory. """ if not strategy.reports_copyright_drift(): return files = _upstream_files(group) actual_key = _upstream_holder_key(files) if actual_key is None: return declared = set(group.declared_copyright_key) actual = set(actual_key) added = actual - declared removed = declared - actual if not added and not removed: return dump = (group.stored.dump() or "") if group.stored else "" delta.add( DeltaEntry( kind=COPYRIGHT_DRIFT, detail=dump, paragraph=dump, was_copyright=group.declared_copyright_key, now_copyright=actual_key, sub_kind=_drift_sub_kind(added, removed), holder_sources=freeze_holder_sources(file_policy.holder_sources(files)), ) ) def _deletion_culprits( paragraph: dc.FilesParagraph, deleted: tuple[str, ...] ) -> tuple[str, ...]: """The deleted paths that would have matched this paragraph's patterns. Pure fnmatch against the declared patterns, so it needs neither the tree nor git; empty when no deleted path matches (nothing to attribute). """ return tuple( sorted( path for path in deleted if any(fnmatch.fnmatch(path, pattern) for pattern in paragraph.files) ) ) def record_stale_paragraph( delta: DeltaReport, paragraph: dc.FilesParagraph, deleted: tuple[str, ...] = () ) -> None: """Record a Files paragraph whose declared patterns matched no files. In changed mode the deleted paths that would have matched are attached as provenance; None when no deleted path matches or when the deleted set is empty (full/partial runs pass none). """ dump = paragraph.dump() or "" license_name = t.cast(dc.License, paragraph.license).synopsis culprits = _deletion_culprits(paragraph, deleted) delta.add( DeltaEntry( kind=STALE_PARAGRAPH, detail=dump, paragraph=dump, paragraph_license=license_name, deleted_files=culprits or None, ) ) def record_stale_patterns( delta: DeltaReport, paragraph: dc.FilesParagraph, matched_patterns: list[str], ) -> None: """Record each declared pattern that matched nothing, deduped per pattern. `set(declared) - set(matched)` yields each dead pattern once, so the paragraph-level and pattern-level views never double-count the same pattern. """ dump = paragraph.dump() or "" license_name = t.cast(dc.License, paragraph.license).synopsis dead = set(paragraph.files) - set(matched_patterns) for pattern in paragraph.files: if pattern in dead: delta.add( DeltaEntry( kind=STALE_PATTERN, detail=dump, pattern=pattern, paragraph_license=license_name, ) ) decopy-0.3.2/decopy/fstree/000077500000000000000000000000001522176660400155635ustar00rootroot00000000000000decopy-0.3.2/decopy/fstree/__init__.py000066400000000000000000000000001522176660400176620ustar00rootroot00000000000000decopy-0.3.2/decopy/fstree/archives.py000066400000000000000000000216411522176660400177450ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Archive expansion: populate the file tree with the contents of archives. Walks tar/zip archives (and archives nested inside them, up to a configured depth) and materialises their members as VirtualFileInfo leaves under ArchiveDirInfo nodes in the tree built by tree.py. """ from __future__ import annotations import collections.abc as abc import contextlib import functools import tarfile import tempfile import zipfile from pathlib import Path from typing import TYPE_CHECKING import xdg.Mime from decopy.common.log import logger from decopy.fstree.nodes import ArchiveDirInfo, DirInfo, VirtualFileInfo from decopy.fstree.parsers import ARCHIVE_MIMES if TYPE_CHECKING: from decopy.common.core_types import Options # Importing zipfile_zstd (if installed) monkeypatches the stdlib zipfile to # decode zstd-compressed members. Without it those members raise # NotImplementedError when read; see warn_unsupported_compression. with contextlib.suppress(ImportError): import zipfile_zstd # type: ignore[import-untyped] # noqa: F401 @functools.cache def warn_unsupported_compression() -> None: """Warn, once per run, that an archive uses an undecodable compression. Cached so a tree full of such members logs a single actionable hint instead of one line per member. Reset with .cache_clear() in tests. """ logger.warning( "Some archive members use a compression method the standard library " "cannot decode (e.g. zstd) and were skipped; install the optional " "'zipfile-zstd' package to read them." ) def _navigate_or_create( start: DirInfo, dir_parts: tuple[str, ...], options: Options ) -> DirInfo | None: """Navigate into dir_parts from start, creating DirInfo nodes as needed. Returns None if any component is excluded or collides with an existing file. """ parent = start for part in dir_parts: if options.exclude_directory_re.search(part): return None sub = parent.get_or_create_subdir(part) if sub is None: return None # name collision with a file parent = sub return parent def _add_virtual_member( archive_dir: ArchiveDirInfo, member_path: str, content: bytes, options: Options, ) -> None: parts = Path(member_path).parts if not parts: return basename = parts[-1] if options.exclude_file_re.search(basename): return parent = _navigate_or_create(archive_dir, parts[:-1], options) if parent is None: return parent.file_list[basename] = VirtualFileInfo( parent=parent, name=basename, content=content ) def _iter_tar_members( archive_path: str, options: Options ) -> abc.Iterator[tuple[str, bytes]]: with tarfile.open(archive_path, mode="r:*") as tf: for member in tf.getmembers(): if not member.isfile() and not member.islnk(): continue if member.size > options.max_archive_member_size: logger.debug( "Skipping oversized member %s (%d B)", member.name, member.size ) continue fobj = tf.extractfile(member) if fobj is None: continue yield member.name, fobj.read() def _iter_zip_members( archive_path: str, options: Options ) -> abc.Iterator[tuple[str, bytes]]: with zipfile.ZipFile(archive_path, "r") as zf: for info in zf.infolist(): if info.is_dir(): continue if info.file_size > options.max_archive_member_size: logger.debug( "Skipping oversized member %s (%d B)", info.filename, info.file_size, ) continue try: content = zf.read(info.filename) except NotImplementedError: warn_unsupported_compression() logger.debug( "Skipping member %s with unsupported compression in %s", info.filename, archive_path, ) continue yield info.filename, content type _Opener = abc.Callable[[str, Options], abc.Iterator[tuple[str, bytes]]] # Ordered container handlers: (exception meaning "not this format", opener). # A new container (rar, 7z, ...) is added as another entry here rather than a # new except arm in iter_archive_members. _ARCHIVE_OPENERS: tuple[tuple[type[Exception], _Opener], ...] = ( (tarfile.TarError, _iter_tar_members), (zipfile.BadZipFile, _iter_zip_members), ) def iter_archive_members( archive_path: str, options: Options ) -> abc.Iterator[tuple[str, bytes]]: """Yield (member_path, content) for each regular file in a supported archive. Tries each opener in _ARCHIVE_OPENERS in order and uses the first whose format matches. Raises the last opener's error (e.g. zipfile.BadZipFile, or OSError from any opener) when no handler recognises the file. """ last_error: Exception | None = None for not_this_format, opener in _ARCHIVE_OPENERS: try: yield from opener(archive_path, options) except not_this_format as error: last_error = error else: return if last_error is not None: raise last_error def _expand_nested_member( archive_dir: ArchiveDirInfo, member_path: str, content: bytes, options: Options, depth: int, ) -> None: """Expand an archive member that is itself an archive, at the given depth.""" parts = Path(member_path).parts if not parts: return basename = parts[-1] if options.exclude_file_re.search(basename): return parent = _navigate_or_create(archive_dir, parts[:-1], options) if parent is None: return nested_dir = ArchiveDirInfo(parent=parent, name=basename, archive_path=member_path) suffix = Path(basename).suffix tmp_path: str | None = None try: with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp: tmp.write(content) tmp_path = tmp.name expand_archive(tmp_path, nested_dir, options, depth) parent.file_list[basename] = nested_dir except (tarfile.TarError, zipfile.BadZipFile, OSError) as e: logger.warning("Failed to expand nested archive %s: %s", member_path, e) parent.file_list[basename] = VirtualFileInfo( parent=parent, name=basename, content=content ) finally: if tmp_path is not None: with contextlib.suppress(OSError): Path(tmp_path).unlink() def expand_archive( archive_path: str, archive_dir: ArchiveDirInfo, options: Options, depth: int, ) -> None: """Populate archive_dir with VirtualFileInfo leaves from the archive. Raises tarfile.TarError, zipfile.BadZipFile, or OSError on corrupt input. """ for member_path, content in iter_archive_members(archive_path, options): if ( str(xdg.Mime.get_type(member_path)) in ARCHIVE_MIMES and depth + 1 < options.max_archive_depth ): _expand_nested_member(archive_dir, member_path, content, options, depth + 1) else: _add_virtual_member(archive_dir, member_path, content, options) def expand_subtree( dirinfo: DirInfo, fs_root: str, options: Options, depth: int ) -> None: """Replace every archive file under dirinfo with its expanded ArchiveDirInfo.""" for name in list(dirinfo.file_list.keys()): subdir = dirinfo.get_subdir(name) if subdir is not None: expand_subtree(subdir, fs_root, options, depth) continue item = dirinfo.file_list[name] full_path = str(Path(fs_root) / item.fullname) if str(xdg.Mime.get_type(full_path)) not in ARCHIVE_MIMES: continue archive_dir = ArchiveDirInfo(parent=dirinfo, name=name, archive_path=full_path) try: expand_archive(full_path, archive_dir, options, depth) dirinfo.file_list[name] = archive_dir except (tarfile.TarError, zipfile.BadZipFile, OSError) as e: logger.warning("Failed to expand archive %s: %s", full_path, e) decopy-0.3.2/decopy/fstree/ignore.py000066400000000000000000000121251522176660400174210ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Decide which tree paths the maintainer has excluded from the source. Combines the debian/copyright Files-Excluded repack list (always honoured) with the VCS ignore lists (.gitignore, .git/info/exclude; gated by use_gitignore). The filesystem walk consults is_ignored to prune nodes before they enter the tree, so generation and explain both skip excluded files. """ from __future__ import annotations import typing as t from pathlib import Path import debian.copyright as dc from decopy.common.log import logger if t.TYPE_CHECKING: from pathspec import GitIgnoreSpec else: try: from pathspec import GitIgnoreSpec except ImportError: GitIgnoreSpec = None _warned_no_pathspec = False def _warn_no_pathspec() -> None: """Warn once that gitignore support is inert without pathspec (DP7).""" global _warned_no_pathspec # noqa: PLW0603 if not _warned_no_pathspec: logger.warning( "pathspec is not installed; .gitignore/.git/info/exclude are ignored " "(install python3-pathspec to honour them)" ) _warned_no_pathspec = True def _ancestors(relpath: str) -> list[str]: """relpath and every ancestor directory, deepest first. A Files-Excluded pattern naming a directory must exclude its whole subtree, so a path is tested against the pattern set as itself and as each of its parent directories (mk-origtargz removes the directory recursively). """ parts = relpath.split("/") return ["/".join(parts[: i + 1]) for i in range(len(parts) - 1, -1, -1)] def _relative_to(match_path: str, anchor: str) -> str | None: """match_path relative to a .gitignore at anchor, or None if out of scope. The root anchor ("") applies everywhere; a nested anchor applies only to its strict descendants (a dir's .gitignore does not judge the dir itself). """ if not anchor: return match_path prefix = anchor + "/" if match_path.startswith(prefix): return match_path[len(prefix) :] return None class IgnoreMatcher: """Answers whether a tree-relative path is excluded from the source.""" def __init__( self, *, use_gitignore: bool, files_excluded: tuple[str, ...], root: str ) -> None: self.use_gitignore = use_gitignore and GitIgnoreSpec is not None self.root = root self._files_excluded = ( dc.globs_to_re(files_excluded) if files_excluded else None ) # (anchor dir, spec) registered shallow-first, so deepest-last wins. self._git_specs: list[tuple[str, GitIgnoreSpec]] = [] if use_gitignore and GitIgnoreSpec is None: _warn_no_pathspec() if self.use_gitignore: self._load_info_exclude() def _load_info_exclude(self) -> None: """Seed the root-anchored, lowest-precedence .git/info/exclude (DP6).""" git = Path(self.root) / ".git" exclude = git / "info" / "exclude" if git.is_dir() and exclude.is_file(): self.add_gitignore("", exclude.read_text(encoding="utf-8").splitlines()) def add_gitignore(self, reldir: str, lines: list[str]) -> None: """Register a directory's .gitignore patterns, anchored at reldir.""" if self.use_gitignore: # use_gitignore is False when GitIgnoreSpec is None, so it is set here. spec = GitIgnoreSpec.from_lines(lines) self._git_specs.append((reldir, spec)) def is_ignored(self, relpath: str, *, is_dir: bool) -> bool: return self._is_files_excluded(relpath) or self._is_git_ignored( relpath, is_dir=is_dir ) def _is_files_excluded(self, relpath: str) -> bool: if self._files_excluded is None: return False return any(self._files_excluded.fullmatch(path) for path in _ancestors(relpath)) def _is_git_ignored(self, relpath: str, *, is_dir: bool) -> bool: # A directory pattern (foo/) only matches a trailing-slash path in git. match_path = relpath + "/" if is_dir else relpath ignored = False for anchor, spec in self._git_specs: rel = _relative_to(match_path, anchor) if rel is None: continue result = spec.check_file(rel) if result.include is not None: ignored = result.include return ignored decopy-0.3.2/decopy/fstree/nodes.py000066400000000000000000000311051522176660400172450ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Filesystem tree node types: the data structures decopy builds and reads. FileInfo/DirInfo (and the ArchiveDirInfo/VirtualFileInfo variants) hold the per-file licensing and copyright data plus the tree shape. The orchestration that builds and processes the tree lives in tree.py; the policy that derives a group identity from a node lives in file_policy.py. """ from __future__ import annotations import collections.abc as abc import typing as t from concurrent import futures from pathlib import Path from decopy.common.core_types import CopyrightHolder, Options from decopy.common.log import logger from decopy.fstree.parsers import ParseSpec, parse_from_spec from decopy.licensing.holders import CopyrightHolders if t.TYPE_CHECKING: from decopy.grouping.group_contracts import LicenseKeyed class FileInfo: def __init__(self, parent: DirInfo | None, name: str) -> None: self.parent: DirInfo | None = parent self.name: str = name self.licenses: set[str] = set() self.license_filenames: set[str] = set() # Group from the existing debian/copyright file. Only its license key is # read (in file_policy.license_key), so it depends on LicenseKeyed # rather than the full GroupProtocol. self.existing_group: LicenseKeyed | None = None # The pattern in the debian/copyright that matches this file self.matching_pattern: str | None = None self.copyrights: CopyrightHolders = CopyrightHolders() # Store whether file was included in the files/dirs to process self._included: bool | None = None def __str__(self) -> str: return self.fullname @property def fullname(self) -> str: # Plain string join, not pathlib: this is hot (called once per node per # grouping/matching/render pass) and Path() construction dominated the # profile. Equivalent to str(Path(parent) / name) for single-component # names, including the empty-parent (root child) case. if self.parent: parent_full = self.parent.fullname return f"{parent_full}/{self.name}" if parent_full else self.name return self.name @property def parsed_license(self) -> str: return " or ".join(sorted(self.licenses)) @property def effective_license(self) -> str: """The license this node contributes when resolving a file's license. For a file it is the raw parsed expression; DirInfo overrides this to mark the license as inherited by the directory's contents. """ return self.parsed_license def add_copyrights(self, copyrights: abc.Iterable[CopyrightHolder]) -> None: self.copyrights.extend(copyrights) def add_licenses( self, licenses: abc.Iterable[str], license_filenames: abc.Iterable[str] | None = None, ) -> None: """Add one or more detected licenses for this file. Args: licenses: a set of license keys. license_filenames: for licenses stored in separate files, the filenames hold the license information. """ self.licenses.update(licenses) if license_filenames: self.license_filenames.update(license_filenames) def get_licenses(self) -> set[str]: if self.licenses or not self.parent: return self.licenses return self.parent.get_licenses() def get_license_filenames(self) -> set[str]: if self.licenses or not self.parent: return self.license_filenames return self.parent.get_license_filenames() @property def included(self) -> bool: if self._included is not None: return self._included if self.parent: self._included = self.parent.included return self._included return False @included.setter def included(self, value: bool | None) -> None: self._included = value def is_excluded(self, options: Options) -> bool: return bool( options.exclude_fullname_re.match(self.fullname) or options.exclude_special_re.match(self.fullname) ) def is_split_boundary(self, options: Options) -> bool: """Whether this node starts a new splitting group. A file never does; it delegates to its containing directory. DirInfo overrides this. """ _ = options return False def is_packaging(self) -> bool: """Whether this node is the debian/ packaging dir or lives under it. debian/copyright is the authoritative record for packaging files, so explain must not derive drift/relicense signals from scanning them. Keys off fullname, so one base method covers both files and directories. """ return self.fullname == "debian" or self.fullname.startswith("debian/") def wildcard_pattern(self) -> str: """The DEP-5 Files pattern that stands for this node. A file is named literally; DirInfo overrides this to wildcard its contents. """ return self.fullname def as_container(self) -> DirInfo | None: """This node viewed as a directory, or None if it is not one. Lets callers obtain the container without testing the node's type; a file is never a container, DirInfo returns itself. """ return None @property def is_container(self) -> bool: """Whether this node holds other nodes (i.e. is a directory).""" return self.as_container() is not None @property def children(self) -> abc.Iterable[FileInfo]: """The node's direct children. A file has none; DirInfo overrides.""" return () def get_child(self, name: str) -> FileInfo | None: """The direct child named `name`, or None. A file has no children; DirInfo overrides.""" _ = name return None def walk(self) -> abc.Iterator[FileInfo]: """Yield this node, then every descendant (pre-order).""" yield self for child in self.children: yield from child.walk() def walk_files(self) -> abc.Iterator[FileInfo]: """Yield every leaf (non-container) node in this subtree, pre-order.""" for node in self.walk(): if not node.is_container: yield node def recompute_total(self) -> int: """Number of leaf files in this subtree. A file is one; DirInfo sums its children.""" return 1 def parse_spec(self, root: str) -> ParseSpec: """A picklable description of how to parse this node (for worker procs).""" return ("file", str(Path(root) / self.fullname), b"") def process_licenses(self, root: str, options: Options) -> FileInfo | None: """Parse the license and copyright information for this file.""" if self.is_excluded(options): logger.debug("Ignored file %s (by --exclude)", self.name) return None copyrights, licenses = parse_from_spec(self.parse_spec(root), options) logger.debug("Adding copyrights %s, to %s", copyrights, self) self.add_copyrights(copyrights) logger.debug("Adding licenses %s, to %s", licenses, self) self.add_licenses(licenses) return self class DirInfo(FileInfo): def __init__(self, parent: DirInfo | None = None, name: str = "") -> None: super().__init__(parent, name) self.file_list: dict[str, FileInfo | DirInfo] = {} # Total number of files in sub tree self.total: int = 0 self.tasks: set[futures.Future[FileInfo | None]] = set() @property def effective_license(self) -> str: if self.licenses: return "Inherited({})".format(" or ".join(sorted(self.licenses))) return "" def is_split_boundary(self, options: Options) -> bool: """A directory starts a new splitting group at the tree root, where it holds a license under --split-on-license, or for "debian" under --split-debian. """ return bool( (options.split_on_license and self.licenses) or not self.parent or (options.split_debian and self.fullname == "debian") ) def wildcard_pattern(self) -> str: """A directory stands for everything under it: "dir/*" (or "*" at the tree root, where fullname is empty).""" if self.fullname: return self.fullname + "/*" return "*" def add( self, path: abc.Sequence[str], dirs: abc.Sequence[str], files: abc.Sequence[str], ) -> None: # Update the total number of files self.total += len(dirs) + len(files) if path: sub = self.get_or_create_subdir(path[0]) # sub is None only when a file already occupies the name, which does # not happen on a real filesystem walk; skip rather than descend. if sub is not None: sub.add(path[1:], dirs, files) return for name in dirs: if name not in self.file_list: self.file_list[name] = DirInfo(self, name) for name in files: if name not in self.file_list: self.file_list[name] = FileInfo(self, name) def as_container(self) -> DirInfo | None: return self @property def children(self) -> abc.Iterable[FileInfo]: return self.file_list.values() def get_child(self, name: str) -> FileInfo | None: return self.file_list.get(name) def get_subdir(self, name: str) -> DirInfo | None: """The child named `name` if it is a subdirectory, else None.""" child = self.file_list.get(name) return child.as_container() if child is not None else None def get_or_create_subdir(self, name: str) -> DirInfo | None: """The subdirectory named `name`, creating it if absent. Returns None if `name` is already taken by a file. """ child = self.file_list.get(name) if child is None: sub = DirInfo(self, name) self.file_list[name] = sub return sub return child.as_container() def __iter__(self) -> abc.Iterator[FileInfo]: return self.walk() def collect_child_licenses(self) -> DirInfo: """Aggregate the licenses found in this directory's license-file children. Drains the futures queued in self.tasks (populated by RootInfo.process), recording the licenses and filenames of any child that parsed as a license file. DirInfo inherits FileInfo.process_licenses, which parses a single file, but never runs it: RootInfo.process only submits leaf files for parsing. """ dir_licenses: set[str] = set() filenames: set[str] = set() for task in futures.as_completed(self.tasks): fileinfo = task.result() if not fileinfo: continue licenses = fileinfo.licenses if not licenses: # Not a license? continue dir_licenses.update(licenses) filenames.add(fileinfo.fullname) self.tasks = set() if dir_licenses: self.add_licenses(dir_licenses, filenames) return self def recompute_total(self) -> int: self.total = sum(child.recompute_total() for child in self.children) return self.total class ArchiveDirInfo(DirInfo): def __init__(self, parent: DirInfo, name: str, archive_path: str) -> None: super().__init__(parent, name) self.archive_path: str = archive_path class VirtualFileInfo(FileInfo): def __init__(self, parent: DirInfo, name: str, content: bytes) -> None: super().__init__(parent, name) self._content: bytes = content def parse_spec(self, root: str) -> ParseSpec: _ = root return ("bytes", self.name, self._content) decopy-0.3.2/decopy/fstree/parsers.py000077500000000000000000000632231522176660400176250ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """File Parsers""" from __future__ import annotations import codecs import collections import collections.abc as abc import contextlib import dataclasses import functools import html import io import re as reo import shutil import string import subprocess import tempfile import tokenize import warnings from pathlib import Path from typing import TYPE_CHECKING import xdg.Mime from decopy.common._regex import rex from decopy.common.log import logger from decopy.common.re_types import ParseResult # Local modules from decopy.licensing.holders import parse_holders from decopy.licensing.licenses import clean_comments, find_licenses if TYPE_CHECKING: from decopy.common.core_types import Options # initialize xdg cache before using it xdg.Mime.update_cache() PRINTABLE_BYTES: bytes = bytes(string.printable, encoding="ascii") type ParserFunc = abc.Callable[[str], ParseResult] type ContentParserFunc = abc.Callable[[str, bytes], ParseResult] @dataclasses.dataclass(frozen=True) class Parser: """A MIME format's two entry points, paired in a single registry row. `from_path` parses a file on disk; `from_bytes` parses in-memory bytes (an archive member). Pairing them means adding a format touches one entry in KNOWN_PARSERS rather than two parallel tables kept in sync by hand. """ from_path: ParserFunc from_bytes: ContentParserFunc # A picklable description of what to parse, so a worker process can parse a node # without holding the node itself: ("file", path, b"") for a file on disk, or # ("bytes", name, content) for an in-memory archive member. type ParseSpec = tuple[str, str, bytes] def has_control_chars(bytes_seq: bytes) -> bool: for i in bytes_seq: # ascii control chars (C0), and delete (DEL) if i < 32 or i == 127: # noqa: PLR2004 # except tab (HT), line feed (LF) and carriage return (CR) if i in (9, 10, 13): continue return True return False def _bytes_reader(data: bytes) -> io.BufferedReader: return io.BufferedReader(io.BytesIO(data)) def detect_bom(buffer: bytes) -> str: # This is what chardet.detect does, but chardet fallsback to ascii. :/ # So you either need to feed all the file to chardet and read it # twice (or keep it in memory). Returns "utf-8" when no BOM is present. for encoding, boms in ( ("UTF-8-SIG", (codecs.BOM_UTF8,)), ("UTF-32", (codecs.BOM_UTF32_LE, codecs.BOM_UTF32_BE)), ("X-ISO-10646-UCS-4-3412", (b"\xfe\xff\x00\x00",)), ("X-ISO-10646-UCS-4-2143", (b"\x00\x00\xff\xfe",)), ("UTF-16", (codecs.BOM_UTF16_LE, codecs.BOM_UTF16_BE)), ): if buffer.startswith(boms): return encoding return "utf-8" def generic_input(reader: io.BufferedReader) -> abc.Iterator[str]: buffer = reader.peek(4) default_encoding = detect_bom(buffer) fallback_encoding = "latin1" for raw_line in reader: try: line = raw_line.decode(default_encoding) except UnicodeDecodeError: if has_control_chars(raw_line): # binary? chars = ( chr(char) if char in PRINTABLE_BYTES else "\n" for char in raw_line ) line = "".join(chars) # Drop extra new lines line = reo.sub(r"\n{2,}", "\n", line) else: line = raw_line.decode(fallback_encoding) # convert mac to unix line = reo.sub(r"\r([^\n])", r"\n\1", line) yield line def _process_content(content: str) -> ParseResult: "Process file contents" holders = parse_holders(content) content = clean_comments(content) licenses = find_licenses(content) return holders, licenses def _generic_parse(reader: io.BufferedReader) -> ParseResult: return _process_content("".join(generic_input(reader))) def generic_parser(filename: str) -> ParseResult: """Generic read, treat all files as text Binary content is filtered ala 'strings(1)'. Bom is detected, files with no bom are treated as utf-8 files, but lines that fail to be decoded are decoded with latin1. """ with Path(filename).open("rb") as f: return _generic_parse(f) def generic_content_parser(_name: str, data: bytes) -> ParseResult: return _generic_parse(_bytes_reader(data)) def _subprocess_output(result: subprocess.CompletedProcess[bytes]) -> str: content = "".join(generic_input(_bytes_reader(result.stdout))) content += "".join(generic_input(_bytes_reader(result.stderr))) return content @functools.lru_cache def cmd_parser_factory(*cmd: str) -> ParserFunc: def _parser(filename: str) -> ParseResult: fullcmd = [*cmd, filename] try: result = subprocess.run( # noqa: S603 fullcmd, check=True, capture_output=True, ) content = _subprocess_output(result) except (subprocess.CalledProcessError, UnicodeDecodeError) as e: logger.info("failed to parse %s with %s, (%s)", filename, cmd, e) logger.info("falling back") return generic_parser(filename) return _process_content(content) if shutil.which(cmd[0]) is None: logger.warning("command %s not found, using generic parser as fallback", cmd[0]) return generic_parser return _parser @functools.lru_cache def cmd_content_parser_factory(*cmd: str) -> ContentParserFunc: if shutil.which(cmd[0]) is None: logger.warning("command %s not found, using generic parser as fallback", cmd[0]) return generic_content_parser def _content_parser(name: str, data: bytes) -> ParseResult: suffix = Path(name).suffix or "" tmp_path: str | None = None try: with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp: tmp_path = tmp.name tmp.write(data) result = subprocess.run( # noqa: S603 [*cmd, tmp_path], capture_output=True, check=True, ) content = _subprocess_output(result) except (OSError, subprocess.CalledProcessError) as e: logger.info("failed to parse %s with %s: %s", name, cmd, e) return generic_content_parser(name, data) finally: if tmp_path is not None: with contextlib.suppress(OSError): Path(tmp_path).unlink() return _process_content(content) return _content_parser def _html_parse(reader: io.BufferedReader, name: str = "") -> ParseResult: # Keep in mind that most template systems will also match this type. We # could use beautifulsoup to obtain the text and the html comments, but we # would loose the templating system comments, and the comments of the js # parts. content = "".join(generic_input(reader)) # Not sure if this should go after the unescape content = reo.sub(r"<\s*br\s*/?>", "\n", content) try: content = html.unescape(content) except Exception as e: logger.info("failed to unescape html entities from %s, (%s)", name, e) logger.info("using content as is") return _process_content(content) def html_parser(filename: str) -> ParseResult: "Drop/convert html entities, and brs" with Path(filename).open("rb") as f: return _html_parse(f, filename) def html_content_parser(name: str, data: bytes) -> ParseResult: return _html_parse(_bytes_reader(data), name) POUND_LINE_RE: rex.Pattern[str] = rex.compile(r"^\s*#") def pound_lines_parser(filename: str) -> ParseResult: """Keep only the lines starting with #""" lines = [] with Path(filename).open("rb") as f: for line in generic_input(f): if POUND_LINE_RE.match(line): lines.append(line) else: lines.append("\n") content = "".join(lines) return _process_content(content) PO_COMMENTED_TRANSLATION_RE: rex.Pattern[str] = rex.compile( r"^\s*#.{,2} msg(?:str|id|ctxt)\b", ) def _po_parse(reader: io.BufferedReader) -> ParseResult: lines = [] for line in generic_input(reader): if not POUND_LINE_RE.match(line) or PO_COMMENTED_TRANSLATION_RE.match(line): lines.append("\n") else: lines.append(line) return _process_content("".join(lines)) def po_file_parser(filename: str) -> ParseResult: """Keep only the comments (ignore also #~ msgid lines)""" with Path(filename).open("rb") as f: return _po_parse(f) def po_content_parser(_name: str, data: bytes) -> ParseResult: return _po_parse(_bytes_reader(data)) def _python_parse(readline: abc.Callable[[], bytes]) -> ParseResult: lines = [] newline = True # We tokenize third-party source only to harvest comments/docstrings; its # own lexical warnings (e.g. invalid escape sequences like f"\{x}") are not # ours to report, so keep them off the user's stderr. with warnings.catch_warnings(): warnings.simplefilter("ignore") for token in tokenize.tokenize(readline): if (token.type == tokenize.COMMENT) or ( newline and token.type == tokenize.STRING ): lines.append(token.string) elif token.type == tokenize.NEWLINE: newline = True elif newline and ( token.type in {tokenize.INDENT, tokenize.NL, tokenize.ENCODING} ): continue else: newline = False return _process_content("\n".join(lines)) def python_parser(filename: str) -> ParseResult: """Extract comments and doc strings from a python file""" with Path(filename).open("rb") as f: return _python_parse(f.readline) def python_content_parser(name: str, data: bytes) -> ParseResult: try: return _python_parse(io.BytesIO(data).readline) except (tokenize.TokenError, SyntaxError): return generic_content_parser(name, data) KNOWN_CMD_PARSERS: dict[str, tuple[str, ...]] = { "application/gzip": ("zcat",), "application/x-bzip2": ("bzcat",), # .bz2 (application/x-bzip was a dead alias) "application/x-bzip3": ("bzip3", "-d"), # .bz3 "application/x-lzma": ("xzcat",), # .lzma "application/x-xz": ("xzcat",), # .xz "application/zstd": ("zstdcat",), # .zst "application/x-lzip": ("lzip", "-d", "-c"), # .lz "image/svg+xml-compressed": ("zcat",), # "application/bzip2": ("exiftool",), # "application/json": ("exiftool",), # "application/octet-stream": ("exiftool",), # "application/postscript": ("exiftool",), # "application/rdf+xml": ("exiftool",), # "application/x-bittorrent": ("exiftool",), # "application/x-gzip": ("exiftool",), # "application/x-httpd-php": ("exiftool",), # "application/x-rar-compressed": ("exiftool",), # "application/x-tar": ("exiftool",), # "application/xml": ("exiftool",), # "application/zip": ("exiftool",), # "image/svg+xml": ("exiftool",), # "text/calendar": ("exiftool",), # "text/html": ("exiftool",), # "text/rtf": ("exiftool",), # "text/vcard": ("exiftool",), "application/ResEdit": ("exiftool",), "application/dicom": ("exiftool",), "application/itunes": ("exiftool",), "application/msword": ("exiftool",), "application/mxf": ("exiftool",), "application/pdf": ("exiftool",), "application/sketch": ("exiftool",), "application/vnd.adobe.fla": ("exiftool",), "application/vnd.adobe.illustrator": ("exiftool",), "application/vnd.adobe.indesign-idml-package": ("exiftool",), "application/vnd.adobe.photoshop": ("exiftool",), "application/vnd.iccprofile": ("exiftool",), "application/vnd.ms-excel": ("exiftool",), "application/vnd.ms-excel.addin.macroEnabled": ("exiftool",), "application/vnd.ms-excel.sheet.binary.macroEnabled": ("exiftool",), "application/vnd.ms-excel.sheet.macroEnabled": ("exiftool",), "application/vnd.ms-excel.template.macroEnabled": ("exiftool",), "application/vnd.ms-officetheme": ("exiftool",), "application/vnd.ms-powerpoint": ("exiftool",), "application/vnd.ms-powerpoint.presentation.macroEnabled": ("exiftool",), "application/vnd.ms-powerpoint.slideshow.macroEnabled": ("exiftool",), "application/vnd.ms-powerpoint.template.macroEnabled": ("exiftool",), "application/vnd.ms-word.document.macroEnabled": ("exiftool",), "application/vnd.ms-word.template.macroEnabledTemplate": ("exiftool",), "application/vnd.oasis.opendocument.chart": ("exiftool",), "application/vnd.oasis.opendocument.database": ("exiftool",), "application/vnd.oasis.opendocument.formula": ("exiftool",), "application/vnd.oasis.opendocument.graphics": ("exiftool",), "application/vnd.oasis.opendocument.image": ("exiftool",), "application/vnd.oasis.opendocument.presentation": ("exiftool",), "application/vnd.oasis.opendocument.spreadsheet": ("exiftool",), "application/vnd.oasis.opendocument.text": ("exiftool",), "application/vnd.openxmlformats-officedocument.presentationml.presentation": ( "exiftool", ), "application/vnd.openxmlformats-officedocument.presentationml.slideshow": ( "exiftool", ), "application/vnd.openxmlformats-officedocument.presentationml.template": ( "exiftool", ), "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": ("exiftool",), "application/vnd.openxmlformats-officedocument.spreadsheetml.template": ( "exiftool", ), "application/vnd.openxmlformats-officedocument.wordprocessingml.document": ( "exiftool", ), "application/vnd.openxmlformats-officedocument.wordprocessingml.template": ( "exiftool", ), "application/vnd.palm": ("exiftool",), "application/vnd.rn-realmedia": ("exiftool",), "application/vnd.rn-realmedia-vbr": ("exiftool",), "application/x-3ds": ("exiftool",), "application/x-captureone": ("exiftool",), "application/x-chm": ("exiftool",), "application/x-dfont": ("exiftool",), "application/x-font-otf": ("exiftool",), "application/x-font-ttf": ("exiftool",), "application/x-font-type1": ("exiftool",), "application/x-indesign": ("exiftool",), "application/x-indesign-interchange": ("exiftool",), "application/x-iso9660-image": ("exiftool",), "application/x-iwork-keynote-sffkey": ("exiftool",), "application/x-iwork-numbers-sffnumbers": ("exiftool",), "application/x-iwork-pages-sffpages": ("exiftool",), "application/x-magick-image": ("exiftool",), "application/x-matroska": ("exiftool",), "application/x-mie": ("exiftool",), "application/x-mobipocket-ebook": ("exiftool",), "application/x-shockwave-flash": ("exiftool",), "application/x-visio": ("exiftool",), "application/x-wmf": ("exiftool",), "audio/audible": ("exiftool",), "audio/flac": ("exiftool",), "audio/mpeg": ("exiftool",), "audio/ogg": ("exiftool",), "audio/x-aiff": ("exiftool",), "audio/x-ds2": ("exiftool",), "audio/x-dss": ("exiftool",), "audio/x-matroska": ("exiftool",), "audio/x-monkeys-audio": ("exiftool",), "audio/x-ms-wma": ("exiftool",), "audio/x-musepack": ("exiftool",), "audio/x-pn-realaudio": ("exiftool",), "audio/x-pn-realaudio-plugin": ("exiftool",), "font/ttf": ("exiftool",), "image/apng": ("exiftool",), "image/bmp": ("exiftool",), "image/bpg": ("exiftool",), "image/flif": ("exiftool",), "image/gif": ("exiftool",), "image/jng": ("exiftool",), "image/jp2": ("exiftool",), "image/jpeg": ("exiftool",), "image/jpm": ("exiftool",), "image/jpx": ("exiftool",), "image/pgf": ("exiftool",), "image/pict": ("exiftool",), "image/png": ("exiftool",), "image/tiff": ("exiftool",), "image/vnd.djvu": ("exiftool",), "image/vnd.fpx": ("exiftool",), "image/vnd.ms-photo": ("exiftool",), "image/vnd.radiance": ("exiftool",), "image/x-adobe-dng": ("exiftool",), "image/x-canon-cr2": ("exiftool",), "image/x-canon-cr3": ("exiftool",), "image/x-canon-crw": ("exiftool",), "image/x-dpx": ("exiftool",), "image/x-epson-erf": ("exiftool",), "image/x-exr": ("exiftool",), "image/x-exv": ("exiftool",), "image/x-fujifilm-raf": ("exiftool",), "image/x-gopro-gpr": ("exiftool",), "image/x-hasselblad-3fr": ("exiftool",), "image/x-hasselblad-fff": ("exiftool",), "image/x-j2c": ("exiftool",), "image/x-kodak-dcr": ("exiftool",), "image/x-kodak-k25": ("exiftool",), "image/x-kodak-kdc": ("exiftool",), "image/x-leica-rwl": ("exiftool",), "image/x-light-lri": ("exiftool",), "image/x-lytro-lfp": ("exiftool",), "image/x-mamiya-mef": ("exiftool",), "image/x-minolta-mrw": ("exiftool",), "image/x-nikon-nef": ("exiftool",), "image/x-nikon-nrw": ("exiftool",), "image/x-olympus-orf": ("exiftool",), "image/x-paintshoppro": ("exiftool",), "image/x-panasonic-rw2": ("exiftool",), "image/x-pentax-pef": ("exiftool",), "image/x-photo-cd": ("exiftool",), "image/x-portable-bitmap": ("exiftool",), "image/x-portable-graymap": ("exiftool",), "image/x-portable-pixmap": ("exiftool",), "image/x-quicktime": ("exiftool",), "image/x-raw": ("exiftool",), "image/x-rawzor": ("exiftool",), "image/x-samsung-srw": ("exiftool",), "image/x-sigma-x3f": ("exiftool",), "image/x-sony-arw": ("exiftool",), "image/x-sony-sr2": ("exiftool",), "image/x-sony-srf": ("exiftool",), "image/x-tiff-big": ("exiftool",), "image/x-xcf": ("exiftool",), "video/3gp": ("exiftool",), "video/3gpp": ("exiftool",), "video/3gpp2": ("exiftool",), "video/divx": ("exiftool",), "video/m2ts": ("exiftool",), "video/mng": ("exiftool",), "video/mp4": ("exiftool",), "video/mpeg": ("exiftool",), "video/ogg": ("exiftool",), "video/quicktime": ("exiftool",), "video/vnd.rn-realvideo": ("exiftool",), "video/webm": ("exiftool",), "video/x-canon-crm": ("exiftool",), "video/x-dv": ("exiftool",), "video/x-flv": ("exiftool",), "video/x-matroska": ("exiftool",), "video/x-ms-asf": ("exiftool",), "video/x-ms-dvr": ("exiftool",), "video/x-ms-wmv": ("exiftool",), "video/x-ms-wtv": ("exiftool",), "video/x-red-r3d": ("exiftool",), } # The generic strings-style parser, used as the fallback and for archives (which # both entry points read as text). A single instance so identity checks work. GENERIC = Parser(from_path=generic_parser, from_bytes=generic_content_parser) # Formats with a dedicated parser pair (path parser + bytes parser). _FORMAT_PARSERS: dict[str, Parser] = { "text/x-gettext-translation": Parser(po_file_parser, po_content_parser), "text/x-gettext-translation-template": Parser(po_file_parser, po_content_parser), "text/x-po": Parser(po_file_parser, po_content_parser), "text/x-pot": Parser(po_file_parser, po_content_parser), "text/x-python": Parser(python_parser, python_content_parser), "text/html": Parser(html_parser, html_content_parser), "application/x-php": Parser(html_parser, html_content_parser), } # Archive containers: both entry points fall back to the generic parser. ARCHIVE_MIMES: frozenset[str] = frozenset( { "application/x-compressed-tar", # .tar.gz / .tgz "application/x-xz-compressed-tar", # .tar.xz "application/x-bzip2-compressed-tar", # .tar.bz2 "application/x-bzip1-compressed-tar", # .tar.bz (bzip1) "application/x-bzip3-compressed-tar", # .tar.bz3 "application/x-zstd-compressed-tar", # .tar.zst "application/x-lzip-compressed-tar", # .tar.lz "application/x-lzma-compressed-tar", # .tar.lzma "application/x-tar", # .tar "application/zip", # .zip / .jar / .war } ) # The single MIME -> Parser registry. Each row carries both entry points, so a # new format is added in exactly one place. KNOWN_PARSERS: dict[str, Parser] = { mime: Parser(cmd_parser_factory(*cmd), cmd_content_parser_factory(*cmd)) for mime, cmd in KNOWN_CMD_PARSERS.items() } KNOWN_PARSERS.update(_FORMAT_PARSERS) KNOWN_PARSERS.update(dict.fromkeys(ARCHIVE_MIMES, GENERIC)) def _mime_bfs(xdg_type: xdg.Mime.MIMEtype) -> abc.Iterator[str]: queue: collections.deque[xdg.Mime.MIMEtype] = collections.deque([xdg_type]) seen: set[str] = set() while queue: xdg_mime = queue.popleft() mime = str(xdg_mime) if mime not in seen: seen.add(mime) yield mime queue.extend(xdg_mime.inherits_from()) logger.debug("Original type: %s, queue: %s", xdg_type, queue) def lookup_parser(xdg_type: xdg.Mime.MIMEtype) -> Parser: """The Parser for a MIME type (walking its inheritance), or GENERIC. Single source of dispatch: the path- and bytes-side lookups below are views of this, so both entry points always agree on which format a MIME resolves to. """ for mime in _mime_bfs(xdg_type): parser = KNOWN_PARSERS.get(mime) if parser is not None: return parser return GENERIC def lookup_mimetypes(xdg_type: xdg.Mime.MIMEtype) -> ParserFunc: """The path-side parser for a MIME type (a view of lookup_parser).""" return lookup_parser(xdg_type).from_path # When no specialised parser/tool matches, the generic parser does strings-style # extraction. Running that over a binary file yields garbage holders, so skip a # binary file at the fallback. "Binary" = a NUL byte in the first chunk, except a # BOM-declared multi-byte text encoding (UTF-16/32/UCS-4 legitimately contain # NULs; detect_bom recognises them and generic_input decodes accordingly). _BINARY_SNIFF_BYTES = 8192 # Empty licenses list, for the "nothing parsed" results below. _NO_LICENSES: list[str] = [] def _is_binary_bytes(data: bytes) -> bool: if detect_bom(data[:4]) != "utf-8": return False return b"\x00" in data[:_BINARY_SNIFF_BYTES] def _path_is_binary(fullname: str) -> bool: try: with Path(fullname).open("rb") as f: return _is_binary_bytes(f.read(_BINARY_SNIFF_BYTES)) except OSError: return False def parse_file(fullname: str, options: Options) -> ParseResult: """Parses the received file with the matching parser. Returns: The list of copyrights and licenses obtained by the parser. """ if options.text: parser: ParserFunc = generic_parser else: xdg_type = xdg.Mime.get_type(fullname) logger.debug("Type for %s is: %s", fullname, xdg_type) parser = lookup_mimetypes(xdg_type) if parser is generic_parser and _path_is_binary(fullname): logger.debug("Skipping binary file %s (no specialised parser)", fullname) return [], _NO_LICENSES copyrights, licenses = parser(fullname) logger.debug("Parsed %s: %s, %s", fullname, copyrights, licenses) return copyrights, licenses def detect_mime_by_magic(header: bytes) -> str | None: if header.startswith(b"#!"): shebang = header[:80].split(b"\n")[0] if b"python" in shebang: return "text/x-python" if b"perl" in shebang: return "text/x-perl" if header[:5] == b" ContentParserFunc: """The bytes-side parser for a MIME type (a view of lookup_parser).""" return lookup_parser(xdg_type).from_bytes def parse_content_bytes(name: str, data: bytes, options: Options) -> ParseResult: """Parse copyright/license info from in-memory bytes. Returns: The list of copyrights and licenses obtained by the parser. """ if options.text: return generic_content_parser(name, data) xdg_type = xdg.Mime.get_type(name) if str(xdg_type) == "application/octet-stream": magic_mime = detect_mime_by_magic(data[:512]) if magic_mime: xdg_type = xdg.Mime.MIMEtype(*magic_mime.split("/", 1)) parser = lookup_content_parser(xdg_type) if parser is generic_content_parser and _is_binary_bytes(data): logger.debug("Skipping binary content %s (no specialised parser)", name) return [], _NO_LICENSES logger.debug("Content parser for %s is: %s", name, parser) copyrights, licenses = parser(name, data) logger.debug("Parsed content %s: %s, %s", name, copyrights, licenses) return copyrights, licenses def parse_from_spec(spec: ParseSpec, options: Options) -> ParseResult: """Parse a node from its picklable ParseSpec (file path or in-memory bytes). Lets a worker process parse without the node object; the kind tag is data, not a type check on the (unavailable) node. """ kind, name, content = spec if kind == "file": return parse_file(name, options) return parse_content_bytes(name, content, options) decopy-0.3.2/decopy/fstree/tree.py000077500000000000000000000242441522176660400171050ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Filesystem tree building and processing (orchestration over decopy.fstree.nodes).""" from __future__ import annotations import collections.abc as abc import functools import os import typing as t from concurrent import futures from pathlib import Path from decopy.common._regex import rex from decopy.common.core_types import CopyrightHolder, Options from decopy.common.log import logger from decopy.fstree.ignore import IgnoreMatcher from decopy.fstree.nodes import DirInfo, FileInfo from decopy.fstree.parsers import ParseSpec, parse_from_spec try: from tqdm import tqdm except ImportError: def tqdm[T](iterable: T, *a: t.Any, **kw: t.Any) -> T: # type: ignore[no-redef] # noqa: ANN401 _ = a, kw return iterable _KNOWN_LICENSE_FILENAMES = (r"^COPYING", r"^COPYRIGHT$", r"^LICENSE") # Below this many files, parse in-process: a ProcessPool's fork+pickle overhead # would outweigh the gain (and keeps the test suite off subprocesses). _PARALLEL_MIN = 200 # One parsed file's result: (index into the item list, copyrights, licenses). type _ParseOutcome = tuple[int, list[CopyrightHolder], list[str]] def _parse_one(options: Options, indexed_spec: tuple[int, ParseSpec]) -> _ParseOutcome: """Parse one (index, spec); empty result on any error. `options` is bound via functools.partial so it reaches worker processes without a module global, and a single malformed file (e.g. a `.py` raising tokenize.TokenError) is skipped rather than aborting the whole run. """ index, spec = indexed_spec try: copyrights, licenses = parse_from_spec(spec, options) except Exception as exc: # a bad file is skipped, not fatal logger.debug("Failed to parse %s: %s", spec, exc) return index, [], [] return index, copyrights, list(licenses) class RootInfo(DirInfo): def __init__(self, root: str) -> None: super().__init__() self.root: str = root @property def names(self) -> abc.Iterator[str]: for item in self: yield str(item) def __getitem__(self, key: str) -> RootInfo | DirInfo | FileInfo: if not key or key == ".": return self current: FileInfo = self for path in key.split(os.path.sep): child = current.get_child(path) if child is None: msg = f"{key} not found" raise IndexError(msg) current = child return current @staticmethod def build(options: Options) -> RootInfo: tree = RootInfo(root=options.root) matcher = IgnoreMatcher( use_gitignore=options.use_gitignore, files_excluded=options.files_excluded, root=options.root, ) for walk_entry in tqdm( os.walk(options.root, topdown=True), desc="Building tree", unit="dir", dynamic_ncols=True, disable=(options.debug or not options.progress), ): tree._scan_dir(walk_entry, matcher, options) if options.expand_archives: # Local import: archives.py imports tree node types at module level, # so importing it here (not at module scope) breaks the cycle. from decopy.fstree.archives import expand_subtree # noqa: PLC0415 expand_subtree(tree, options.root, options, depth=0) tree.recompute_total() tree.tag_included_files(options) return tree def _scan_dir( self, walk_entry: tuple[str, list[str], list[str]], matcher: IgnoreMatcher, options: Options, ) -> None: """Prune one os.walk directory's children, then add the survivors. Pruning happens in place on the dirs list so os.walk (topdown) does not descend into a removed directory. The dir's own .gitignore is registered first, so it applies to the children pruned here. """ root, dirs, files = walk_entry logger.debug("Listing %s", root) reldir = root[len(self.root) + 1 :] root_path = Path(root) self._register_gitignore(matcher, root_path, reldir) def relpath(name: str) -> str: return f"{reldir}/{name}" if reldir else name def keep_dir(name: str) -> bool: if options.exclude_directory_re.search(name): logger.debug("Ignoring directory %s as requested", name) return False if matcher.is_ignored(relpath(name), is_dir=True): logger.debug("Ignoring directory %s (ignore list)", name) return False return True def keep_file(name: str) -> bool: if options.exclude_file_re.search(name): logger.debug("Ignoring file %s as requested", name) return False if (root_path / name).is_symlink(): logger.debug("Ignoring symlink %s", name) return False if matcher.is_ignored(relpath(name), is_dir=False): logger.debug("Ignoring file %s (ignore list)", name) return False return True dirs[:] = [name for name in dirs if keep_dir(name)] files[:] = [name for name in files if keep_file(name)] self.add(reldir.split(os.path.sep) if reldir else [], dirs, files) def _register_gitignore( self, matcher: IgnoreMatcher, root_path: Path, reldir: str ) -> None: """Register this directory's .gitignore, anchored at reldir, if present. Reads from disk (not the pruned file list) since .gitignore is itself excluded by name yet must still inform pruning. """ if not matcher.use_gitignore: return gitignore = root_path / ".gitignore" if gitignore.is_file(): matcher.add_gitignore( reldir, gitignore.read_text(encoding="utf-8").splitlines() ) def tag_included_files(self, options: Options) -> None: """Determine which files to include, from the specified cmd options.""" if not options.files and not options.scoped_to_subset: # A scoped run (partial/changed) with no files is an empty scope, # not the whole tree; only an unscoped run defaults to everything. self.included = True for filename in options.files: # Special case, if the root directory was received as a positional # parameter, include the whole tree. if filename == ".": self.included = True return # Filenames not in the tree will cause an exception. Catch that and # log an error in that case. try: self[filename].included = True except IndexError: logger.error("%s not in source tree", filename) def process(self, options: Options) -> None: """Parse every included file's licenses/copyrights, then aggregate directory licenses. Big trees parse in a process pool: the per-file work is GIL-bound (regex subs, holder parsing), so threads do not help; processes scale. Workers return results and the parent applies them - they cannot mutate the shared tree. """ items = [ item for item in self.walk_files() if item.included and item.parent is not None and not item.is_excluded(options) ] self._apply_results(items, self._parse_items(items, options)) def _parse_items( self, items: list[FileInfo], options: Options ) -> list[_ParseOutcome]: """Parse every item, in a process pool for big trees (the per-file work is GIL-bound), serially otherwise. Same worker either way.""" specs = [(i, item.parse_spec(self.root)) for i, item in enumerate(items)] worker = functools.partial(_parse_one, options) workers = options.jobs or (os.cpu_count() or 1) def progress(iterable: abc.Iterable[t.Any]) -> abc.Iterable[t.Any]: return tqdm( iterable, desc="Processing", total=len(specs), unit="file", dynamic_ncols=True, disable=(options.debug or not options.progress), ) if workers > 1 and len(specs) >= _PARALLEL_MIN: chunk = max(1, len(specs) // (workers * 4)) with futures.ProcessPoolExecutor(max_workers=workers) as executor: return list(progress(executor.map(worker, specs, chunksize=chunk))) return [worker(spec) for spec in progress(specs)] def _apply_results( self, items: list[FileInfo], results: list[_ParseOutcome] ) -> None: dirs_with_licenses: set[DirInfo] = set() for index, copyrights, licenses in results: item = items[index] item.add_copyrights(copyrights) item.add_licenses(licenses) parent = item.parent if parent is None: continue for filename_re in _KNOWN_LICENSE_FILENAMES: if rex.match(filename_re, item.name): resolved: futures.Future[FileInfo | None] = futures.Future() resolved.set_result(item) parent.tasks.add(resolved) dirs_with_licenses.add(parent) break for dirinfo in dirs_with_licenses: dirinfo.collect_child_licenses() decopy-0.3.2/decopy/fstree/vcs.py000066400000000000000000000130421522176660400167300ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Compute the files changed since debian/copyright was last committed. A leaf module: it shells out to git and returns plain data, owning no policy. On any unavailability (git missing, not a repo, an untracked copyright file, a bad --since ref) it returns a Fallback carrying the reason, so the caller can warn accurately while deciding whether to degrade to a full scan. """ from __future__ import annotations import dataclasses import shutil import subprocess from decopy.common.log import logger GIT_MISSING = "git-missing" NOT_A_REPO = "not-a-repo" COPYRIGHT_UNTRACKED = "copyright-untracked" BAD_REF = "bad-ref" @dataclasses.dataclass(frozen=True) class ChangedPaths: """Files that differ from the baseline, classified by how they changed.""" added_or_modified: tuple[str, ...] deleted: tuple[str, ...] renamed: tuple[tuple[str, str], ...] @dataclasses.dataclass(frozen=True) class Fallback: """A reason the changed set could not be computed, with a ready message.""" reason: str message: str def _fallback( reason: str, *, root: str = "", copyright_path: str = "", since: str = "" ) -> Fallback: messages = { GIT_MISSING: "git was not found on PATH; scanning the full tree " "(install git or drop --changed)", NOT_A_REPO: f"{root} is not a git repository; scanning the full tree", COPYRIGHT_UNTRACKED: f"{copyright_path} is not tracked in git; " "scanning the full tree", BAD_REF: f"--since {since} is not a valid git ref; scanning the full tree", } return Fallback(reason=reason, message=messages[reason]) def _git(root: str, *args: str) -> str | None: """Run one git command in `root`; stdout on success, None on any failure. FileNotFoundError (git absent) and PermissionError are OSError subclasses, so a missing or unrunnable git collapses to None here rather than escaping. """ cmd = ["git", "-C", root, *args] try: result = subprocess.run( # noqa: S603 cmd, check=True, capture_output=True, text=True, ) except (OSError, subprocess.CalledProcessError) as exc: logger.debug("git %s failed: %s", args, exc) return None return result.stdout def _git_available() -> bool: return shutil.which("git") is not None def _resolve_base(root: str, copyright_path: str, since: str) -> str | None: """The baseline commit sha, or None when it cannot be resolved. With --since it is that ref; otherwise the last commit that touched the copyright file (empty output means the file has no git history). """ if since: out = _git(root, "rev-parse", "--verify", f"{since}^{{commit}}") else: out = _git(root, "log", "-n1", "--format=%H", "--", copyright_path) if out is None: return None sha = out.strip() return sha or None def _parse_name_status( text: str, ) -> tuple[list[str], list[str], list[tuple[str, str]]]: """Split `git diff --name-status -M` output into (added_or_modified, deleted, renamed) by each line's status code.""" added: list[str] = [] deleted: list[str] = [] renamed: list[tuple[str, str]] = [] for line in text.splitlines(): if not line: continue fields = line.split("\t") code = fields[0][0] if code == "R": renamed.append((fields[1], fields[2])) elif code == "D": deleted.append(fields[1]) else: added.append(fields[1]) return added, deleted, renamed def _untracked(root: str) -> list[str]: out = _git(root, "ls-files", "--others", "--exclude-standard") return out.splitlines() if out else [] def changed_paths( root: str, copyright_path: str, since: str = "" ) -> ChangedPaths | Fallback: """The files changed since the baseline, or a Fallback with the reason. The baseline is the commit that last touched `copyright_path` (or `since` when given). Working-tree changes are diffed against it and untracked files are added; paths are git's repo-root-relative form. """ if not _git_available(): return _fallback(GIT_MISSING) if _git(root, "rev-parse", "--is-inside-work-tree") is None: return _fallback(NOT_A_REPO, root=root) base = _resolve_base(root, copyright_path, since) if base is None: reason = BAD_REF if since else COPYRIGHT_UNTRACKED return _fallback(reason, copyright_path=copyright_path, since=since) added, deleted, renamed = _parse_name_status( _git(root, "diff", "--name-status", "-M", base) or "" ) changed = set(added) | set(_untracked(root)) return ChangedPaths( added_or_modified=tuple(sorted(changed)), deleted=tuple(sorted(deleted)), renamed=tuple(renamed), ) decopy-0.3.2/decopy/grouping/000077500000000000000000000000001522176660400161255ustar00rootroot00000000000000decopy-0.3.2/decopy/grouping/__init__.py000066400000000000000000000000001522176660400202240ustar00rootroot00000000000000decopy-0.3.2/decopy/grouping/file_policy.py000066400000000000000000000121601522176660400207750ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """File-to-group policy: derive a group identity from a tree node. These functions *read* a tree node (FileInfo/DirInfo) to compute the license key, splitting path, and group key, rather than that policy living on the node itself. Keeping it here lets the tree carry data and shape only. """ from __future__ import annotations import typing as t from collections import abc from decopy.common.core_types import UNKNOWN, UNKNOWN_COPYRIGHTED from decopy.grouping.grouping import GroupKey, make_group_key from decopy.grouping.grouping_strategy import grouping_strategy_for if t.TYPE_CHECKING: from decopy.common.core_types import Options from decopy.fstree.nodes import FileInfo def license_key(node: FileInfo) -> str: """Resolve the license identifier for a node, in precedence order: 1. An explicit (non-wildcarded) entry in debian/copyright. 2. The license detected when parsing the file. 3. The wildcarded license stored in debian/copyright. 4. The license of the containing directory (recursively). Falls back to UNKNOWN_COPYRIGHTED when copyrights are present, else UNKNOWN. """ if node.existing_group and node.fullname == node.matching_pattern: return node.existing_group.get_license_key() if node.effective_license: return node.effective_license if node.existing_group: return node.existing_group.get_license_key() if node.parent: return license_key(node.parent) if node.copyrights: return UNKNOWN_COPYRIGHTED return UNKNOWN def _from_files(filenames: set[str]) -> str: """Provenance phrase naming the license file(s) a license came from.""" return "from {}".format(", ".join(sorted(filenames))) def _detected_source(node: FileInfo) -> str | None: """Phrase for a node's own detected license, or None when it has none.""" if not node.licenses: return None if node.license_filenames: return _from_files(node.license_filenames) return f"detected in {node.fullname}" def _inherited_source(node: FileInfo | None) -> str | None: """Phrase for a license inherited from the nearest ancestor that holds one, or None when no ancestor contributes a license. A license from a separate file names that file (more precise than the directory).""" while node is not None: if node.licenses: if node.license_filenames: return _from_files(node.license_filenames) return f"inherited from {node.fullname}/" node = node.parent return None def license_source(node: FileInfo) -> str: """Describe where a node's license came from, mirroring license_key's precedence: the file's own detection, a separate license file, an ancestor directory, else none. Names a file or directory, so --spdx never applies. """ own = _detected_source(node) if own is not None: return own inherited = _inherited_source(node.parent) if inherited is not None: return inherited if node.copyrights: return "copyright present, no license" return "no license detected" def holder_sources(files: abc.Iterable[FileInfo]) -> dict[str, tuple[str, ...]]: """Map each copyright holder to the sorted files it was detected in. The key is the holder's ``email or person``, matching CopyrightHolders.key(), so it lines up with the holder strings the explain +/- lines render. The caller chooses which files to pass (e.g. excluding packaging files). """ sources: dict[str, set[str]] = {} for fileinfo in files: for holder in fileinfo.copyrights: key = holder.email or holder.person sources.setdefault(key, set()).add(fileinfo.fullname) return {key: tuple(sorted(paths)) for key, paths in sources.items()} def splitting_path(node: FileInfo, options: Options) -> str | None: """Path identifier used to split files into groups. Delegates to the active grouping strategy, which owns the --split-on-license / --split-debian rules. """ return grouping_strategy_for(options.group_by).splitting_path(node, options) def group_key(node: FileInfo, options: Options) -> GroupKey: """Assemble the (license, copyright, path) group key for a node.""" return make_group_key( license_key(node), node.copyrights.key(), splitting_path(node, options), options, ) decopy-0.3.2/decopy/grouping/group.py000066400000000000000000000271671522176660400176500ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Group management and processing""" from __future__ import annotations import collections.abc as abc import os import typing as t import debian.copyright as dc from decopy.common.core_types import ( UNKNOWN, UNKNOWN_COPYRIGHTED, CopyrightHolder, License, Options, ) from decopy.fstree.nodes import FileInfo from decopy.grouping.file_policy import splitting_path from decopy.grouping.group_contracts import GroupProtocol from decopy.grouping.grouping import GroupKey, make_group_key from decopy.grouping.grouping_strategy import grouping_strategy_for from decopy.licensing.holders import CopyrightHolders from decopy.licensing.license_expr import ( Node, parse_dep5_expression, render_dep5_expression, render_spdx_expression, ) # The DEP-5 Files-paragraph fields the renderer emits from a group's own data; # any other field a stored paragraph carried is passed through via extra_fields. _STANDARD_FILES_FIELDS = frozenset({"Files", "Copyright", "License", "Comment"}) class TallyTracker: """Tracks nodes already emitted under a wildcard. When file paragraphs are wildcarded, a later, more specific group must not re-wildcard an ancestor an earlier group already covered. One tracker is shared across all groups in a single output run; it holds the bookkeeping that used to live on the nodes themselves. """ def __init__(self) -> None: self._tallied: set[FileInfo] = set() def first_untallied(self, node: FileInfo) -> FileInfo: """The topmost ancestor of `node` (including itself) not yet tallied.""" if node.parent is not None and node.parent not in self._tallied: return self.first_untallied(node.parent) return node def tally(self, node: FileInfo) -> None: """Mark `node` and all its ancestors as tallied.""" current: FileInfo | None = node while current is not None and current not in self._tallied: self._tallied.add(current) current = current.parent class FileGroup: def __init__(self) -> None: self.files: dict[str, FileInfo] = {} def __len__(self) -> int: return len(self.files) def __bool__(self) -> bool: return bool(self.files) def __iter__(self) -> abc.Iterator[FileInfo]: return iter(self.files.values()) def __repr__(self) -> str: return f"FileGroup({self.sorted_members()!s})" def add_file(self, fileinfo: FileInfo) -> None: self.files[fileinfo.fullname] = fileinfo def del_file(self, fileinfo: FileInfo) -> FileInfo: return self.files.pop(fileinfo.fullname) def get_patterns(self, tracker: TallyTracker) -> list[str]: """Returns the wildcarded patters for the contained files. The order in which this method is called is important. It should be called with the group that will get the more general wildcard first, and with the groups that will get more specific values later. The shared tracker carries that ordering state across groups. """ patterns: set[str] = set() to_tally: set[FileInfo] = set() for fileinfo in self: untallied = tracker.first_untallied(fileinfo) to_tally.add(untallied) for fileinfo in to_tally: patterns.add(fileinfo.wildcard_pattern()) for fileinfo in self: tracker.tally(fileinfo) return sorted(patterns) def path_key(self, options: Options) -> str: if not self.files: return "" return os.path.commonpath( tuple(t.cast(str, splitting_path(v, options)) for v in self.files.values()), ) def commonpath(self) -> str: if not self.files: return "" return os.path.commonpath(tuple(f.fullname for f in self.files.values())) def sorted_members(self) -> list[str]: return sorted(self.files.keys()) class Group(GroupProtocol): """Group class for managing files and their associated copyright/license information """ def __init__( self, key: GroupKey | None = None, position: int = 0, ) -> None: self.key: GroupKey | None = key self.files: FileGroup = FileGroup() self.copyrights: CopyrightHolders = CopyrightHolders() # The holders parsed from an existing paragraph's Copyright field, # snapshotted before any file holders are merged in. Explain mode # compares it against the kept files' holders to detect drift. self.declared_copyright_key: tuple[str, ...] = () self.licenses: dict[str, License] = {} # The parsed licence expression when this group was read from an existing # paragraph, kept so the field can be re-emitted faithfully (and/or/comma # structure). None for groups decopy synthesises from a detected set. self.license_expr: Node | None = None self.license_filenames: set[str] = set() self.comments: list[str] = [] self.stored: dc.FilesParagraph | None = None self.position: int = position @property def license(self) -> str: """The canonical license key for this group (used for grouping/matching). Connector-agnostic: the detected set joined with " or " and sorted, so a file and its existing paragraph produce the same key. The output License field is `license_field`, not this. """ name = " or ".join(sorted(self.licenses)) if not name: if self.copyrights: return UNKNOWN_COPYRIGHTED return UNKNOWN return name @property def license_field(self) -> str: """The License field text to render for this group. When the group was read from an existing paragraph, render from the preserved expression so and/or/comma structure round-trips; fall back to an SPDX expression when DEP-5 cannot represent the tree without changing its meaning. Synthesised groups (no expression) use `license`. """ if self.license_expr is None: return self.license dep5 = render_dep5_expression(self.license_expr) if parse_dep5_expression(dep5) == self.license_expr: return dep5 return render_spdx_expression(self.license_expr) def __str__(self) -> str: return str(self.key) if self.key else self.license def __repr__(self) -> str: return f"Group({self!s})" def add_file(self, fileinfo: FileInfo) -> None: """Add a file to this group and update associated metadata.""" self.files.add_file(fileinfo) self.copyrights.merge(fileinfo.copyrights) for name in fileinfo.get_licenses(): self.licenses[name] = License(name) self.license_filenames.update(fileinfo.get_license_filenames()) def add_copyrights(self, copyrights: abc.Iterable[CopyrightHolder]) -> None: """Add copyright information to this group.""" self.copyrights.extend(copyrights) def get_inherited_comment(self) -> str | None: """Get inherited comment for license files.""" if not self.license_filenames: return None # A group read from an existing paragraph already carries its own # comment; don't synthesise the "No explicit license found" note on top # of it. if self.stored: return None filenames = sorted(self.license_filenames) return "No explicit license found, using license(s) from:\n {}".format( "\n ".join(filenames), ) def get_comments(self) -> str: """Get all comments for this group.""" comments: list[str] = [] inherited_comment = self.get_inherited_comment() if inherited_comment: comments.append(inherited_comment) comments.extend(self.comments) return "\n .\n ".join(comments) def extra_fields(self) -> abc.Iterator[tuple[str, str]]: """Yield the stored paragraph's non-standard (name, value) fields. The standard DEP-5 Files-paragraph fields are rendered by the renderer from the group's own data; any other field an existing paragraph carried (e.g. an X-* field) is surfaced here, so the renderer depends on this accessor rather than on the concrete debian paragraph type. """ if not self.stored: return # Group owns the concrete dc.FilesParagraph, so the paragraph-shaped # access lives here (the renderer sees only this accessor). The stubs # type __iter__ as Iterable[str], hence the ignore on the for-loop. for field in self.stored: # type: ignore[attr-defined] # ty: ignore[not-iterable] if field in _STANDARD_FILES_FIELDS: continue yield field, self.stored[field] def get_license_key(self) -> str: """Get the license key for this group.""" return self.license def get_copyright_key(self) -> tuple[str, ...]: """Get the copyright key for this group.""" return self.copyrights.key() def get_path_key(self, options: Options) -> str: """Get the path key for this group.""" return self.files.path_key(options) def get_key( self, options: Options, *, ignore_copyright: bool = False, ) -> GroupKey: """The group's key under `options`. With `ignore_copyright` the copyright slot is blanked, so two groups that differ only by their holders compare equal (used when deciding whether a file is merely misplaced rather than relicensed). The full key is computed once and cached on `self.key`. `options` is invariant for a Group's lifetime (one Options object per run), so the cache never has to notice a changed `options`. """ if not self.key: self.key = make_group_key( self.get_license_key(), self.get_copyright_key(), self.get_path_key(options), options, ) if ignore_copyright: return self.key._replace(copyright=()) return self.key def copyright_block_valid(self) -> bool: """Check if this group can generate a valid copyright block.""" if not self.files: return False if len(self.files) > 1: return True return bool(next(f.fullname for f in self.files)) def sort_key( self, options: Options, ) -> tuple[int, int, int, int, t.Any]: strategy = grouping_strategy_for(options.group_by) debian_group = strategy.sort_band( self.license, self.get_path_key(options), split_debian=options.split_debian ) return ( debian_group, -1 * self.position, -1 * len(self.files), -1 * len(self.copyrights), self.get_key(options, ignore_copyright=True), ) decopy-0.3.2/decopy/grouping/group_contracts.py000066400000000000000000000066601522176660400217230ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """ Group interface and data contracts extracted from dep5.py This module defines the interfaces and data structures that Group classes must implement, enabling better separation of concerns and testing. """ from __future__ import annotations import collections.abc as abc import typing as t import debian.copyright as dc from decopy.common.core_types import CopyrightHolder, License, Options if t.TYPE_CHECKING: from decopy.fstree.nodes import FileInfo from decopy.grouping.group import FileGroup from decopy.grouping.grouping import GroupKey from decopy.licensing.holders import CopyrightHolders from decopy.licensing.license_expr import Node @t.runtime_checkable class LicenseKeyed(t.Protocol): """Something that can report a license key - a Group, seen from a file. This is the only part of a group a FileInfo needs from its existing_group, so that field depends on this rather than on the full GroupProtocol. """ def get_license_key(self) -> str: """Get the license key.""" ... @t.runtime_checkable class GroupProtocol(LicenseKeyed, t.Protocol): """ Protocol defining the interface that all Group classes must implement. This protocol allows for different Group implementations while maintaining a consistent interface. """ @property def license(self) -> str: ... @property def license_field(self) -> str: ... license_expr: Node | None key: GroupKey | None position: int files: FileGroup copyrights: CopyrightHolders declared_copyright_key: tuple[str, ...] licenses: dict[str, License] license_filenames: set[str] comments: list[str] stored: dc.FilesParagraph | None def add_file(self, fileinfo: FileInfo) -> None: """Add a file to this group.""" ... def add_copyrights(self, copyrights: abc.Iterable[CopyrightHolder]) -> None: """Add copyrights to this group.""" ... def get_key( self, options: Options, *, ignore_copyright: bool = False, ) -> GroupKey: """Get the key for this group based on options.""" ... def copyright_block_valid(self) -> bool: """Check if this group can generate a valid copyright block.""" ... def get_comments(self) -> str: """Get the assembled comment text for this group.""" ... def extra_fields(self) -> abc.Iterable[tuple[str, str]]: """Yield the stored paragraph's non-standard (name, value) fields.""" ... def sort_key(self, options: Options) -> tuple[int, int, int, int, t.Any]: """Get the sort key for this group.""" ... decopy-0.3.2/decopy/grouping/grouping.py000066400000000000000000000044711522176660400203370ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Shared grouping vocabulary: the group key type and its assembly rule. A group key is the (license, copyright, path) triple that decides which files land in the same copyright paragraph. Both the per-file view (tree.FileInfo) and the per-group view (group.Group) must build it the same way, so the single assembly rule lives here rather than being duplicated in each. """ from __future__ import annotations import typing as t from decopy.grouping.grouping_strategy import grouping_strategy_for if t.TYPE_CHECKING: from decopy.common.core_types import Options class GroupKey(t.NamedTuple): """The (license, copyright, path) triple that decides paragraph membership. A NamedTuple, so the slots read as ``key.license`` / ``key.copyright`` / ``key.path`` rather than ``key[0..2]``, while staying a plain tuple for equality, hashing and use as a dict key. """ license: str copyright: tuple[str, ...] | None path: str | None def make_group_key( license_key: str, copyright_key: tuple[str, ...], path_key: str | None, options: Options, ) -> GroupKey: """Assemble a group key from its parts, applying the grouping policy. The copyright slot is filled only when grouping by copyright; otherwise it is None. Keeping this rule in one place ensures files and groups always agree on the key shape. """ strategy = grouping_strategy_for(options.group_by) return GroupKey( license_key, strategy.key_copyright_slot(copyright_key), path_key, ) decopy-0.3.2/decopy/grouping/grouping_strategy.py000066400000000000000000000146771522176660400222720ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Grouping policy as an injected strategy. How files become copyright paragraphs is an axis decopy is expected to vary (by license today, by copyright today, perhaps by directory or vendor later). This module gathers that policy behind a single GroupingStrategy so the consumers (grouping, file_policy, group, copyright_processor) depend on the abstraction rather than on scattered group_by/split_* conditionals. The two concrete strategies differ only in how copyright participates: * LicenseGrouping drops the copyright slot from the group key and keeps a misplaced file only when its full key matches the group key. * CopyrightGrouping keeps the copyright slot and additionally rescues a file whose key matches the group key with copyright ignored. The split_on_license/split_debian rules (splitting_path) and the sort band are shared, so they live on the base class; a future mode that needs a different boundary or ordering overrides them. The strategy is selected once per consumer via grouping_strategy_for(options): grouping.make_group_key (copyright slot), file_policy.splitting_path (split rules), group.Group.sort_key (sort band) and copyright_processor's remove_misplaced_files (keep/drop fallback) all depend on it. """ from __future__ import annotations import abc import typing as t from decopy.common.core_types import UNKNOWN, UNKNOWN_COPYRIGHTED if t.TYPE_CHECKING: from decopy.common.core_types import Options from decopy.fstree.nodes import FileInfo from decopy.grouping.grouping import GroupKey class GroupingStrategy(abc.ABC): """Owns the policy that turns files into copyright-paragraph groups.""" @abc.abstractmethod def key_copyright_slot( self, copyright_key: tuple[str, ...] ) -> tuple[str, ...] | None: """Return the copyright component of a group key, or None to omit it.""" @abc.abstractmethod def keeps_misplaced( self, file_key: GroupKey, group_key: GroupKey, group_key_no_copyright: GroupKey, ) -> bool: """Whether a file already placed in a group still belongs there.""" def splitting_path(self, node: FileInfo, options: Options) -> str | None: """Path identifier used to split files into groups. A node that is a split boundary (see FileInfo.is_split_boundary) contributes its own path; otherwise the node delegates to its parent. A parentless non-boundary node (a file at the root) has no splitting path. """ if node.is_split_boundary(options): return node.fullname if node.parent: return self.splitting_path(node.parent, options) return None def reports_copyright_drift(self) -> bool: """Whether explain mode should report copyright-holder drift for this mode. Default False: a strategy that keeps the copyright slot in its key already surfaces holder changes as relicensed entries, so reporting drift too would double-count. LicenseGrouping (which masks the slot) overrides this. """ return False def sort_band(self, license_name: str, path_key: str, *, split_debian: bool) -> int: """Primary sort band for a group: 0 normal, 1 unknown, 2 debian. Takes just the `split_debian` flag it needs rather than the whole Options. Unknown licenses override the debian band (they share band 1), matching the historical precedence in Group.sort_key. """ band = 0 if split_debian and path_key == "debian": band = 2 if license_name in {UNKNOWN, UNKNOWN_COPYRIGHTED}: band = 1 return band class LicenseGrouping(GroupingStrategy): """Group by license: copyright never participates in the key.""" def reports_copyright_drift(self) -> bool: # The copyright slot is masked out of the key, so a holder change is # invisible to the relicensed path; drift is the only signal for it. return True def key_copyright_slot( self, copyright_key: tuple[str, ...] ) -> tuple[str, ...] | None: _ = copyright_key return None def keeps_misplaced( self, file_key: GroupKey, group_key: GroupKey, group_key_no_copyright: GroupKey, ) -> bool: _ = group_key_no_copyright return file_key == group_key class CopyrightGrouping(GroupingStrategy): """Group by copyright: the copyright slot participates as a subset relation. A paragraph that groups several files declares the union of their holders in its Copyright field, so each file legitimately carries a subset. A file is kept when its license and path match the group and its holders are a subset of the group's declared holders; the empty-holder case (an uncopyrighted file) is the subset relation's trivial member, so no separate rescue is needed. """ def key_copyright_slot( self, copyright_key: tuple[str, ...] ) -> tuple[str, ...] | None: return copyright_key def keeps_misplaced( self, file_key: GroupKey, group_key: GroupKey, group_key_no_copyright: GroupKey, ) -> bool: _ = group_key_no_copyright if file_key.license != group_key.license or file_key.path != group_key.path: return False return set(file_key.copyright or ()) <= set(group_key.copyright or ()) _LICENSE_GROUPING = LicenseGrouping() _COPYRIGHT_GROUPING = CopyrightGrouping() def grouping_strategy_for(group_by: str) -> GroupingStrategy: """Select the grouping strategy named by `group_by` ("license"/"copyright").""" if group_by == "copyright": return _COPYRIGHT_GROUPING return _LICENSE_GROUPING decopy-0.3.2/decopy/licensing/000077500000000000000000000000001522176660400162465ustar00rootroot00000000000000decopy-0.3.2/decopy/licensing/__init__.py000066400000000000000000000000001522176660400203450ustar00rootroot00000000000000decopy-0.3.2/decopy/licensing/data/000077500000000000000000000000001522176660400171575ustar00rootroot00000000000000decopy-0.3.2/decopy/licensing/data/__init__.py000066400000000000000000000000001522176660400212560ustar00rootroot00000000000000decopy-0.3.2/decopy/licensing/data/common_licenses.py000066400000000000000000000107431522176660400227130ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """DEP-5 licence name -> /usr/share/common-licenses file. Maps the canonical DEP-5 names decopy emits to the base-files common-licenses filename that holds the full text, so a standalone License paragraph can point at the shared copy instead of duplicating it. The +/only distinction and the GFDL invariants choice do not change the referenced text file (only the notice header does), so an "or-later" name resolves to the same entry as its base and the GFDL-NIV (no-invariants) name shares its version's file. """ from __future__ import annotations import typing as t class CommonLicense(t.NamedTuple): file: str # basename under /usr/share/common-licenses friendly_name: str # e.g. "the GNU General Public License version 2" COMMON_LICENSES: dict[str, CommonLicense] = { "GPL-1": CommonLicense("GPL-1", "the GNU General Public License version 1"), "GPL-2": CommonLicense("GPL-2", "the GNU General Public License version 2"), "GPL-3": CommonLicense("GPL-3", "the GNU General Public License version 3"), "LGPL-2": CommonLicense( "LGPL-2", "the GNU Library General Public License version 2" ), "LGPL-2.1": CommonLicense( "LGPL-2.1", "the GNU Lesser General Public License version 2.1" ), "LGPL-3": CommonLicense( "LGPL-3", "the GNU Lesser General Public License version 3" ), "AGPL-3": CommonLicense( "AGPL-3.0", "the GNU Affero General Public License version 3" ), # decopy detection emits GFDL- (invariants allowed) and GFDL-NIV- # (no invariants); both reference the same version's common-licenses text. "GFDL-1.1": CommonLicense( "GFDL-1.1", "the GNU Free Documentation License version 1.1" ), "GFDL-NIV-1.1": CommonLicense( "GFDL-1.1", "the GNU Free Documentation License version 1.1" ), "GFDL-1.2": CommonLicense( "GFDL-1.2", "the GNU Free Documentation License version 1.2" ), "GFDL-NIV-1.2": CommonLicense( "GFDL-1.2", "the GNU Free Documentation License version 1.2" ), "GFDL-1.3": CommonLicense( "GFDL-1.3", "the GNU Free Documentation License version 1.3" ), "GFDL-NIV-1.3": CommonLicense( "GFDL-1.3", "the GNU Free Documentation License version 1.3" ), "Apache-2": CommonLicense("Apache-2.0", "the Apache License, Version 2.0"), "Artistic-1": CommonLicense("Artistic", "the Artistic License"), "Artistic-2": CommonLicense("Artistic-2.0", "the Artistic License version 2.0"), "BSL-1": CommonLicense("BSL-1.0", "the Boost Software License version 1.0"), "CC-BY-3": CommonLicense( "CC-BY-3.0", "the Creative Commons Attribution 3.0 licence" ), "CC-BY-4": CommonLicense( "CC-BY-4.0", "the Creative Commons Attribution 4.0 International Public Licence" ), "CC-BY-SA-3": CommonLicense( "CC-BY-SA-3.0", "the Creative Commons Attribution-ShareAlike 3.0 licence" ), "CC-BY-SA-4": CommonLicense( "CC-BY-SA-4.0", "the Creative Commons Attribution-ShareAlike 4.0 International Public Licence", ), "CC0": CommonLicense("CC0-1.0", "the Creative Commons CC0 1.0 Universal licence"), "MPL-1.1": CommonLicense("MPL-1.1", "the Mozilla Public License version 1.1"), "MPL-2": CommonLicense("MPL-2.0", "the Mozilla Public License version 2.0"), "OFL-1.1": CommonLicense("OFL-1.1", "the SIL Open Font License version 1.1"), } def common_license_for(name: str) -> CommonLicense | None: """The common-licenses entry for a DEP-5 name, or None if it has no copy. Strips a trailing "+" (or-later) so it resolves to the same file as the base name; the GFDL-NIV names are enumerated and matched directly. """ return COMMON_LICENSES.get(name.rstrip("+")) decopy-0.3.2/decopy/licensing/data/common_licenses_index.txt000066400000000000000000000003041522176660400242610ustar00rootroot00000000000000AGPL-3.0 Apache-2.0 Artistic Artistic-2.0 BSL-1.0 CC0-1.0 CC-BY-3.0 CC-BY-4.0 CC-BY-SA-3.0 CC-BY-SA-4.0 GFDL-1.1 GFDL-1.2 GFDL-1.3 GPL-1 GPL-2 GPL-3 LGPL-2 LGPL-2.1 LGPL-3 MPL-1.1 MPL-2.0 OFL-1.1 decopy-0.3.2/decopy/licensing/data/exceptions/000077500000000000000000000000001522176660400213405ustar00rootroot00000000000000decopy-0.3.2/decopy/licensing/data/exceptions/AutoConf-3.txt000066400000000000000000000034321522176660400237610ustar00rootroot00000000000000AUTOCONF CONFIGURE SCRIPT EXCEPTION Version 3.0, 18 August 2009 Copyright © 2009 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. This Exception is an additional permission under section 7 of the GNU General Public License, version 3 ("GPLv3"). It applies to a given file that bears a notice placed by the copyright holder of the file stating that the file is governed by GPLv3 along with this Exception. The purpose of this Exception is to allow distribution of Autoconf's typical output under terms of the recipient's choice (including proprietary). 0. Definitions. "Covered Code" is the source or object code of a version of Autoconf that is a covered work under this License. "Normally Copied Code" for a version of Autoconf means all parts of its Covered Code which that version can copy from its code (i.e., not from its input file) into its minimally verbose, non-debugging and non-tracing output. "Ineligible Code" is Covered Code that is not Normally Copied Code. 1. Grant of Additional Permission. You have permission to propagate output of Autoconf, even if such propagation would otherwise violate the terms of GPLv3. However, if by modifying Autoconf you cause any Ineligible Code of the version you received to become Normally Copied Code of your modified version, then you void this Exception for the resulting covered work. If you convey that resulting covered work, you must remove this Exception in accordance with the second paragraph of Section 7 of GPLv3. 2. No Weakening of Autoconf Copyleft. The availability of this Exception does not imply any general presumption that third-party software is unaffected by the copyleft requirements of the license of Autoconf. decopy-0.3.2/decopy/licensing/data/exceptions/AutoConf-macro.txt000066400000000000000000000014001522176660400247110ustar00rootroot00000000000000As a special exception, the respective Autoconf Macro's copyright owner gives unlimited permission to copy, distribute and modify the configure scripts that are the output of Autoconf when processing the Macro. You need not follow the terms of the GNU General Public License when using or distributing such scripts, even though portions of the text of the Macro appear in them. The GNU General Public License (GPL) does govern all other use of the material that constitutes the Autoconf Macro. This special exception to the GPL applies to versions of the Autoconf Macro released by the Autoconf Archive. When you make and distribute a modified version of the Autoconf Macro, you may extend this special exception to the GPL to apply to your modified version as well. decopy-0.3.2/decopy/licensing/data/exceptions/AutoConf.txt000066400000000000000000000030451522176660400236210ustar00rootroot00000000000000As a special exception, the Free Software Foundation gives unlimited permission to copy, distribute and modify the configure scripts that are the output of Autoconf. You need not follow the terms of the GNU General Public License when using or distributing such scripts, even though portions of the text of Autoconf appear in them. The GNU General Public License (GPL) does govern all other use of the material that constitutes the Autoconf program. Certain portions of the Autoconf source text are designed to be copied (in certain cases, depending on the input) into the output of Autoconf. We call these the "data" portions. The rest of the Autoconf source text consists of comments plus executable code that decides which of the data portions to output in any given case. We call these comments and executable code the "non-data" portions. Autoconf never copies any of the non-data portions into its output. This special exception to the GPL applies to versions of Autoconf released by the Free Software Foundation. When you make and distribute a modified version of Autoconf, you may extend this special exception to the GPL to apply to your modified version as well, *unless* your modified version has the potential to copy into its output some of the text that was the non-data portion of the version that you started with. (In other words, unless your change moves or copies text from the non-data portions to the data portions.) If your modification has such potential, you must delete any notice of this special exception to the GPL from your modified version. decopy-0.3.2/decopy/licensing/data/exceptions/Bison.txt000066400000000000000000000012411522176660400231510ustar00rootroot00000000000000Bison Exception As a special exception, you may create a larger work that contains part or all of the Bison parser skeleton and distribute that work under terms of your choice, so long as that work isn't itself a parser generator using the skeleton or a modified version thereof as a parser skeleton. Alternatively, if you modify or redistribute the parser skeleton itself, you may (at your option) remove this special exception, which will cause the skeleton and the resulting Bison output files to be licensed under the GNU General Public License without this special exception. This special exception was added by the Free Software Foundation in version 2.2 of Bison. decopy-0.3.2/decopy/licensing/data/exceptions/Bootloader.txt000066400000000000000000000010131522176660400241660ustar00rootroot00000000000000Bootloader Exception -------------------- In addition to the permissions in the GNU General Public License, the authors give you unlimited permission to link or embed compiled bootloader and related files into combinations with other programs, and to distribute those combinations without any restriction coming from the use of those files. (The General Public License restrictions do apply in other respects; for example, they cover modification of the files, and distribution when not linked into a combine executable.) decopy-0.3.2/decopy/licensing/data/exceptions/ClassPath.txt000066400000000000000000000016271522176660400237710ustar00rootroot00000000000000Linking this library statically or dynamically with other modules is making a combined work based on this library. Thus, the terms and conditions of the GNU General Public License cover the whole combination. As a special exception, the copyright holders of this library give you permission to link this library with independent modules to produce an executable, regardless of the license terms of these independent modules, and to copy and distribute the resulting executable under terms of your choice, provided that you also meet, for each linked independent module, the terms and conditions of the license of that module. An independent module is a module which is not derived from or based on this library. If you modify this library, you may extend this exception to your version of the library, but you are not obligated to do so. If you do not wish to do so, delete this exception statement from your version. decopy-0.3.2/decopy/licensing/data/exceptions/Font.txt000066400000000000000000000011061522176660400230050ustar00rootroot00000000000000As a special exception, if you create a document which uses this font, and embed this font or unaltered portions of this font into the document, this font does not by itself cause the resulting document to be covered by the GNU General Public License. This exception does not however invalidate any other reasons why the document might be covered by the GNU General Public License. If you modify this font, you may extend this exception to your version of the font, but you are not obligated to do so. If you do not wish to do so, delete this exception statement from your version. decopy-0.3.2/decopy/licensing/data/exceptions/GCC-3.1.txt000066400000000000000000000064761522176660400230110ustar00rootroot00000000000000GCC RUNTIME LIBRARY EXCEPTION Version 3.1, 31 March 2009 General information: http://www.gnu.org/licenses/gcc-exception.html Copyright (C) 2009 Free Software Foundation, Inc. Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. This GCC Runtime Library Exception ("Exception") is an additional permission under section 7 of the GNU General Public License, version 3 ("GPLv3"). It applies to a given file (the "Runtime Library") that bears a notice placed by the copyright holder of the file stating that the file is governed by GPLv3 along with this Exception. When you use GCC to compile a program, GCC may combine portions of certain GCC header files and runtime libraries with the compiled program. The purpose of this Exception is to allow compilation of non-GPL (including proprietary) programs to use, in this way, the header files and runtime libraries covered by this Exception. 0. Definitions. A file is an "Independent Module" if it either requires the Runtime Library for execution after a Compilation Process, or makes use of an interface provided by the Runtime Library, but is not otherwise based on the Runtime Library. "GCC" means a version of the GNU Compiler Collection, with or without modifications, governed by version 3 (or a specified later version) of the GNU General Public License (GPL) with the option of using any subsequent versions published by the FSF. "GPL-compatible Software" is software whose conditions of propagation, modification and use would permit combination with GCC in accord with the license of GCC. "Target Code" refers to output from any compiler for a real or virtual target processor architecture, in executable form or suitable for input to an assembler, loader, linker and/or execution phase. Notwithstanding that, Target Code does not include data in any format that is used as a compiler intermediate representation, or used for producing a compiler intermediate representation. The "Compilation Process" transforms code entirely represented in non-intermediate languages designed for human-written code, and/or in Java Virtual Machine byte code, into Target Code. Thus, for example, use of source code generators and preprocessors need not be considered part of the Compilation Process, since the Compilation Process can be understood as starting with the output of the generators or preprocessors. A Compilation Process is "Eligible" if it is done using GCC, alone or with other GPL-compatible software, or if it is done without using any work based on GCC. For example, using non-GPL-compatible Software to optimize any GCC intermediate representations would not qualify as an Eligible Compilation Process. 1. Grant of Additional Permission. You have permission to propagate a work of Target Code formed by combining the Runtime Library with Independent Modules, even if such propagation would otherwise violate the terms of GPLv3, provided that all Target Code was generated by Eligible Compilation Processes. You may then convey such a combination under terms of your choice, consistent with the licensing of the Independent Modules. 2. No Weakening of GCC Copyleft. The availability of this Exception does not imply any general presumption that third-party software is unaffected by the copyleft requirements of the license of GCC. decopy-0.3.2/decopy/licensing/data/exceptions/GCC.txt000066400000000000000000000007421522176660400225000ustar00rootroot00000000000000In addition to the permissions in the GNU General Public License, the Free Software Foundation gives you unlimited permission to link the compiled version of this file into combinations with other programs, and to distribute those combinations without any restriction coming from the use of this file. (The General Public License restrictions do apply in other respects; for example, they cover modification of the file, and distribution when not linked into a combine executable.) decopy-0.3.2/decopy/licensing/data/exceptions/LLVM.txt000066400000000000000000000016271522176660400226610ustar00rootroot00000000000000---- LLVM Exceptions to the Apache 2.0 License ---- As an exception, if, as a result of your compiling your source code, portions of this Software are embedded into an Object form of such source code, you may redistribute such embedded portions in such Object form without complying with the conditions of Sections 4(a), 4(b) and 4(d) of the License. In addition, if you combine or link compiled forms of this Software with software that is licensed under the GPLv2 ("Combined Software") and if a court of competent jurisdiction determines that the patent provision (Section 3), the indemnity provision (Section 9) or other Section of the License conflicts with the conditions of the GPLv2, you may retroactively and prospectively choose to deem waived or otherwise exclude such Section(s) of the License, but only in their entirety and only with respect to the Combined Software. decopy-0.3.2/decopy/licensing/data/exceptions/Linux-syscall.txt000066400000000000000000000011731522176660400246520ustar00rootroot00000000000000NOTE! This copyright does *not* cover user programs that use kernel services by normal system calls - this is merely considered normal use of the kernel, and does *not* fall under the heading of "derived work". Also note that the GPL below is copyrighted by the Free Software Foundation, but the instance of code that it refers to (the Linux kernel) is copyrighted by me and others who actually wrote it. Also note that the only valid version of the GPL as far as the kernel is concerned is _this_ particular version of the license (ie v2, not v2.2 or v3.x or whatever), unless explicitly otherwise stated. Linus Torvalds decopy-0.3.2/decopy/licensing/data/exceptions/OCaml-LGPL.txt000066400000000000000000000015611522176660400236330ustar00rootroot00000000000000As a special exception to the GNU Lesser General Public License, you may link, statically or dynamically, a "work that uses the OCaml Core System" with a publicly distributed version of the OCaml Core System to produce an executable file containing portions of the OCaml Core System, and distribute that executable file under terms of your choice, without any of the additional requirements listed in clause 6 of the GNU Lesser General Public License. By "a publicly distributed version of the OCaml Core System", we mean either the unmodified OCaml Core System as distributed by INRIA, or a modified version of the OCaml Core System that is distributed under the conditions defined in clause 2 of the GNU Lesser General Public License. This exception does not however invalidate any other reasons why the executable file might be covered by the GNU Lesser General Public License. decopy-0.3.2/decopy/licensing/data/exceptions/PCRE2.txt000066400000000000000000000006351522176660400227200ustar00rootroot00000000000000EXEMPTION FOR BINARY LIBRARY-LIKE PACKAGES ------------------------------------------ The second condition in the BSD licence (covering binary redistributions) does not apply all the way down a chain of software. If binary package A includes PCRE2, it must respect the condition, but if package B is software that includes package A, the condition is not imposed on package B unless it uses PCRE2 independently. decopy-0.3.2/decopy/licensing/data/exceptions/Qt-GPL-1.txt000066400000000000000000000017041522176660400232450ustar00rootroot00000000000000The Qt Company GPL Exception 1.0 Exception 1: As a special exception you may create a larger work which contains the output of this application and distribute that work under terms of your choice, so long as the work is not otherwise derived from or based on this application and so long as the work does not in itself generate output that contains the output from this application in its original or modified form. Exception 2: As a special exception, you have permission to combine this application with Plugins licensed under the terms of your choice, to produce an executable, and to copy and distribute the resulting executable under the terms of your choice. However, the executable must be accompanied by a prominent notice offering all users of the executable the entire source code to this application, excluding the source code of the independent modules, but including any changes you have made to this application, under the terms of this license. decopy-0.3.2/decopy/licensing/data/exceptions/Qt-LGPL-1.1.txt000066400000000000000000000022531522176660400235200ustar00rootroot00000000000000The Qt Company Qt LGPL Exception version 1.1 As an additional permission to the GNU Lesser General Public License version 2.1, the object code form of a "work that uses the Library" may incorporate material from a header file that is part of the Library. You may distribute such object code under terms of your choice, provided that: (i) the header files of the Library have not been modified; and (ii) the incorporated material is limited to numerical parameters, data structure layouts, accessors, macros, inline functions and templates; and (iii) you comply with the terms of Section 6 of the GNU Lesser General Public License version 2.1. Moreover, you may apply this exception to a modified version of the Library, provided that such modification does not involve copying material from the Library into the modified Library's header files unless such material is limited to (i) numerical parameters; (ii) data structure layouts; (iii) accessors; and (iv) small macros, templates and inline functions of five lines or less in length. Furthermore, you are not required to apply this additional permission to a modified version of the Library. decopy-0.3.2/decopy/licensing/data/headers/000077500000000000000000000000001522176660400205725ustar00rootroot00000000000000decopy-0.3.2/decopy/licensing/data/headers/AGPL-3+.txt000066400000000000000000000011701522176660400222700ustar00rootroot00000000000000This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see . decopy-0.3.2/decopy/licensing/data/headers/AGPL-3.txt000066400000000000000000000010731522176660400222170ustar00rootroot00000000000000This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, version 3. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details. You should have received a copy of the GNU Affero General Public License along with this program. If not, see . decopy-0.3.2/decopy/licensing/data/headers/Apache-2.txt000066400000000000000000000010101522176660400226430ustar00rootroot00000000000000Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. decopy-0.3.2/decopy/licensing/data/headers/GFDL-NIV-1.1+.txt000066400000000000000000000005721522176660400230550ustar00rootroot00000000000000Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". decopy-0.3.2/decopy/licensing/data/headers/GFDL-NIV-1.1.txt000066400000000000000000000004731522176660400230020ustar00rootroot00000000000000Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". decopy-0.3.2/decopy/licensing/data/headers/GFDL-NIV-1.2+.txt000066400000000000000000000005571522176660400230610ustar00rootroot00000000000000Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts,and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". decopy-0.3.2/decopy/licensing/data/headers/GFDL-NIV-1.2.txt000066400000000000000000000004611522176660400230000ustar00rootroot00000000000000Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". decopy-0.3.2/decopy/licensing/data/headers/GFDL-NIV-1.3+.txt000066400000000000000000000005601522176660400230540ustar00rootroot00000000000000Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". decopy-0.3.2/decopy/licensing/data/headers/GFDL-NIV-1.3.txt000066400000000000000000000004611522176660400230010ustar00rootroot00000000000000Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License". decopy-0.3.2/decopy/licensing/data/headers/GPL-1+.txt000066400000000000000000000012331522176660400221650ustar00rootroot0000000000000019yy This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. decopy-0.3.2/decopy/licensing/data/headers/GPL-1.txt000066400000000000000000000011521522176660400221120ustar00rootroot0000000000000019xx name of author This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 1. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. decopy-0.3.2/decopy/licensing/data/headers/GPL-2+.txt000066400000000000000000000012431522176660400221670ustar00rootroot00000000000000This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA . decopy-0.3.2/decopy/licensing/data/headers/GPL-2.txt000066400000000000000000000011541522176660400221150ustar00rootroot00000000000000This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA . decopy-0.3.2/decopy/licensing/data/headers/GPL-3+.txt000066400000000000000000000011431522176660400221670ustar00rootroot00000000000000This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . decopy-0.3.2/decopy/licensing/data/headers/GPL-3.txt000066400000000000000000000010461522176660400221160ustar00rootroot00000000000000This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see . decopy-0.3.2/decopy/licensing/data/headers/LGPL-2+.txt000066400000000000000000000012741522176660400223070ustar00rootroot00000000000000This library is free software; you can redistribute it and/or modify it under the terms of the GNU Library General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public License for more details. You should have received a copy of the GNU Library General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. decopy-0.3.2/decopy/licensing/data/headers/LGPL-2.1+.txt000066400000000000000000000012751522176660400224470ustar00rootroot00000000000000This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA decopy-0.3.2/decopy/licensing/data/headers/LGPL-2.1.txt000066400000000000000000000012001522176660400223600ustar00rootroot00000000000000This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; version 2.1. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA decopy-0.3.2/decopy/licensing/data/headers/LGPL-2.txt000066400000000000000000000011771522176660400222360ustar00rootroot00000000000000This library is free software; you can redistribute it and/or modify it under the terms of the GNU Library General Public License as published by the Free Software Foundation; version 2. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Library General Public License for more details. You should have received a copy of the GNU Library General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA. decopy-0.3.2/decopy/licensing/data/headers/MPL-1.1.txt000066400000000000000000000007121522176660400222600ustar00rootroot00000000000000The contents of this file are subject to the Mozilla Public License Version 1.1 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.mozilla.org/MPL/ Software distributed under the License is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for the specific language governing rights and limitations under the License. decopy-0.3.2/decopy/licensing/data/headers/MPL-2.txt000066400000000000000000000003021522176660400221150ustar00rootroot00000000000000This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at https://mozilla.org/MPL/2.0/. decopy-0.3.2/decopy/licensing/data/licenses/000077500000000000000000000000001522176660400207645ustar00rootroot00000000000000decopy-0.3.2/decopy/licensing/data/licenses/0BSD.txt000066400000000000000000000011371522176660400222170ustar00rootroot00000000000000Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted. THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. decopy-0.3.2/decopy/licensing/data/licenses/BSD-1-clause.txt000066400000000000000000000020121522176660400235400ustar00rootroot00000000000000Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS AND CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. decopy-0.3.2/decopy/licensing/data/licenses/BSD-2-clause-FreeBSD.txt000066400000000000000000000026301522176660400247570ustar00rootroot00000000000000Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE FREEBSD PROJECT ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FREEBSD PROJECT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of the FreeBSD Project. decopy-0.3.2/decopy/licensing/data/licenses/BSD-2-clause.txt000066400000000000000000000023241522176660400235470ustar00rootroot00000000000000Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. decopy-0.3.2/decopy/licensing/data/licenses/BSD-3-clause-Clear.txt000066400000000000000000000030561522176660400245770ustar00rootroot00000000000000Redistribution and use in source and binary forms, with or without modification, are permitted (subject to the limitations in the disclaimer below) provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of [Owner Organization] nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. NO EXPRESS OR IMPLIED LICENSES TO ANY PARTY'S PATENT RIGHTS ARE GRANTED BY THIS LICENSE. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. decopy-0.3.2/decopy/licensing/data/licenses/BSD-3-clause.txt000066400000000000000000000026241522176660400235530ustar00rootroot00000000000000Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. decopy-0.3.2/decopy/licensing/data/licenses/BSD-4-clause.txt000066400000000000000000000030361522176660400235520ustar00rootroot00000000000000Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. All advertising materials mentioning features or use of this software must display the following acknowledgement: This product includes software developed by the organization. 4. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY COPYRIGHT HOLDER "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL COPYRIGHT HOLDER BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. decopy-0.3.2/decopy/licensing/data/licenses/CC-PDDC.txt000066400000000000000000000030701522176660400225220ustar00rootroot00000000000000The person or persons who have associated work with this document (the "Dedicator" or "Certifier") hereby either (a) certifies that, to the best of his knowledge, the work of authorship identified is in the public domain of the country from which the work is published, or (b) hereby dedicates whatever copyright the dedicators holds in the work of authorship identified below (the "Work") to the public domain. A certifier, moreover, dedicates any copyright interest he may have in the associated work, and for these purposes, is described as a "dedicator" below. A certifier has taken reasonable steps to verify the copyright status of this work. Certifier recognizes that his good faith efforts may not shield him from liability if in fact the work certified is not in the public domain. Dedicator makes this dedication for the benefit of the public at large and to the detriment of the Dedicator's heirs and successors. Dedicator intends this dedication to be an overt act of relinquishment in perpetuity of all present and future rights under copyright law, whether vested or contingent, in the Work. Dedicator understands that such relinquishment of all rights includes the relinquishment of all rights to enforce (by lawsuit or otherwise) those copyrights in the Work. Dedicator recognizes that, once placed in the public domain, the Work may be freely reproduced, distributed, transmitted, used, modified, built upon, or otherwise exploited by anyone for any purpose, commercial or non-commercial, and in any way, including by methods that have not yet been invented or conceived. decopy-0.3.2/decopy/licensing/data/licenses/Expat.txt000066400000000000000000000017771522176660400226220ustar00rootroot00000000000000Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. decopy-0.3.2/decopy/licensing/data/licenses/FSFAP.txt000066400000000000000000000003411522176660400223620ustar00rootroot00000000000000Copying and distribution of this file, with or without modification, are permitted in any medium without royalty provided the copyright notice and this notice are preserved. This file is offered as-is, without any warranty. decopy-0.3.2/decopy/licensing/data/licenses/FSFUL.txt000066400000000000000000000002031522176660400223770ustar00rootroot00000000000000This configure script is free software; the Free Software Foundation gives unlimited permission to copy, distribute and modify it. decopy-0.3.2/decopy/licensing/data/licenses/FSFULLR.txt000066400000000000000000000002661522176660400226460ustar00rootroot00000000000000This file is free software; the Free Software Foundation gives unlimited permission to copy and/or distribute it, with or without modifications, as long as this notice is preserved. decopy-0.3.2/decopy/licensing/data/licenses/ISC.txt000066400000000000000000000012531522176660400221440ustar00rootroot00000000000000Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. decopy-0.3.2/decopy/licensing/data/licenses/MIT-0.txt000066400000000000000000000015321522176660400223140ustar00rootroot00000000000000Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. decopy-0.3.2/decopy/licensing/data/licenses/PSF-2.txt000066400000000000000000000045151522176660400223210ustar00rootroot000000000000001. This LICENSE AGREEMENT is between the Python Software Foundation ("PSF"), and the Individual or Organization ("Licensee") accessing and otherwise using this software ("Python") in source or binary form and its associated documentation. 2. Subject to the terms and conditions of this License Agreement, PSF hereby grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce, analyze, test, perform and/or display publicly, prepare derivative works, distribute, and otherwise use Python alone or in any derivative version, provided, however, that PSF's License Agreement and PSF's notice of copyright, i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019 Python Software Foundation; All Rights Reserved" are retained in Python alone or in any derivative version prepared by Licensee. 3. In the event Licensee prepares a derivative work that is based on or incorporates Python or any part thereof, and wants to make the derivative work available to others as provided herein, then Licensee hereby agrees to include in any such work a brief summary of the changes made to Python. 4. PSF is making Python available to Licensee on an "AS IS" basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT INFRINGE ANY THIRD PARTY RIGHTS. 5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON, OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF. 6. This License Agreement will automatically terminate upon a material breach of its terms and conditions. 7. Nothing in this License Agreement shall be deemed to create any relationship of agency, partnership, or joint venture between PSF and Licensee. This License Agreement does not grant permission to use PSF trademarks or trade name in a trademark sense to endorse or promote products or services of Licensee, or any third party. 8. By copying, installing or otherwise using Python, Licensee agrees to be bound by the terms and conditions of this License Agreement. decopy-0.3.2/decopy/licensing/data/licenses/Unicode-3.txt000066400000000000000000000035601522176660400232570ustar00rootroot00000000000000NOTICE TO USER: Carefully read the following legal agreement. BY DOWNLOADING, INSTALLING, COPYING OR OTHERWISE USING DATA FILES, AND/OR SOFTWARE, YOU UNEQUIVOCALLY ACCEPT, AND AGREE TO BE BOUND BY, ALL OF THE TERMS AND CONDITIONS OF THIS AGREEMENT. IF YOU DO NOT AGREE, DO NOT DOWNLOAD, INSTALL, COPY, DISTRIBUTE OR USE THE DATA FILES OR SOFTWARE. Permission is hereby granted, free of charge, to any person obtaining a copy of data files and any associated documentation (the "Data Files") or software and any associated documentation (the "Software") to deal in the Data Files or Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Data Files or Software, and to permit persons to whom the Data Files or Software are furnished to do so, provided that either (a) this copyright and permission notice appear with all copies of the Data Files or Software, or (b) this copyright and permission notice appear in associated Documentation. THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA FILES OR SOFTWARE. Except as contained in this notice, the name of a copyright holder shall not be used in advertising or otherwise to promote the sale, use or other dealings in these Data Files or Software without prior written authorization of the copyright holder. decopy-0.3.2/decopy/licensing/data/licenses/Unlicense.txt000066400000000000000000000022731522176660400234560ustar00rootroot00000000000000This is free and unencumbered software released into the public domain. Anyone is free to copy, modify, publish, use, compile, sell, or distribute this software, either in source code form or as a compiled binary, for any purpose, commercial or non-commercial, and by any means. In jurisdictions that recognize copyright laws, the author or authors of this software dedicate any and all copyright interest in the software to the public domain. We make this dedication for the benefit of the public at large and to the detriment of our heirs and successors. We intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this software under copyright law. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. For more information, please refer to decopy-0.3.2/decopy/licensing/data/licenses/X11.txt000066400000000000000000000024141522176660400220770ustar00rootroot00000000000000Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Except as contained in this notice, the name of the X Consortium shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization from the X Consortium. X Window System is a trademark of X Consortium, Inc. decopy-0.3.2/decopy/licensing/data/licenses/Zlib.txt000066400000000000000000000014701522176660400224270ustar00rootroot00000000000000This software is provided 'as-is', without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software. Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions: 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 3. This notice may not be removed or altered from any source distribution. decopy-0.3.2/decopy/licensing/data/licenses/curl.txt000066400000000000000000000016361522176660400225000ustar00rootroot00000000000000Permission to use, copy, modify, and distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. Except as contained in this notice, the name of a copyright holder shall not be used in advertising or otherwise to promote the sale, use or other dealings in this Software without prior written authorization of the copyright holder. decopy-0.3.2/decopy/licensing/data/loader.py000066400000000000000000000064451522176660400210100ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Readers for the shipped license-text data package. The bulk data (the common-licenses index, the notice headers and the embedded licence texts) ships as package-data under this directory and is read via importlib.resources so it works from a wheel as well as a source checkout. """ from __future__ import annotations import functools import importlib.resources @functools.cache def common_licenses_index() -> frozenset[str]: """The base-files common-licenses filenames decopy may reference. A committed snapshot of the real /usr/share/common-licenses/ entries (the GPL/LGPL/GFDL symlinks excluded, explicit versions only), so the map validation and runtime stay hermetic and independent of the host. """ text = ( importlib.resources.files(__package__) .joinpath("common_licenses_index.txt") .read_text(encoding="utf-8") ) return frozenset(line.strip() for line in text.splitlines() if line.strip()) @functools.cache def variant_header(name: str) -> str | None: """The cleaned notice header for a DEP-5 variant name, or None if none ships. Keyed on the full variant name (e.g. "GPL-2" vs "GPL-2+"), since the header is what distinguishes the +/only variant. Returns None when no header file exists for the name (most licences), so the caller emits the reference alone. """ path = importlib.resources.files(__package__).joinpath("headers", f"{name}.txt") if not path.is_file(): return None return path.read_text(encoding="utf-8").strip() or None @functools.cache def embedded_license(name: str) -> str | None: """The embedded full licence text for a DEP-5 name, or None if none ships. Short/popular licences not in common-licenses (Expat, the BSDs, ISC, ...) are shipped as verbatim text; returns it as-is (the caller does not wrap it). """ path = importlib.resources.files(__package__).joinpath("licenses", f"{name}.txt") if not path.is_file(): return None return path.read_text(encoding="utf-8").strip() or None @functools.cache def embedded_exception(label: str) -> str | None: """The embedded exception text for a DEP-5 label, or None if none ships. Curated exception texts (AutoConf, ClassPath, GCC, ...) are shipped as verbatim text; returns it as-is (the caller does not wrap it). """ path = importlib.resources.files(__package__).joinpath("exceptions", f"{label}.txt") if not path.is_file(): return None return path.read_text(encoding="utf-8").strip() or None decopy-0.3.2/decopy/licensing/holders.py000066400000000000000000000530541522176660400202670ustar00rootroot00000000000000#!/usr/bin/env python3 # # Copyright: 2016, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Copyright-holder extraction: parse copyright notices into CopyrightHolders.""" from __future__ import annotations import collections.abc as abc # Local modules from decopy.common._regex import rex from decopy.common.core_types import CopyrightHolder, YearRange from decopy.licensing.license_subs import ( COPYRIGHT_INDICATOR_RE, COPYRIGHT_POST_IGNORE_RES, COPYRIGHT_PRE_IGNORE_RES, COPYRIGHT_PRE_INDICATOR_RE, CRUFT_SUBS, EMAIL_SUBS, NAME_CRUFT_SUBS, ) # Copyright holders handling # We are discarding parts that are shorter than MIN_PART_LEN MIN_PART_LEN: int = 3 # A copyright notice longer than this is not scanned (runaway/binary blob guard). MAX_NOTICE_LEN: int = 4096 # Year-normalisation pivots used by _get_year. _SINGLE_DIGIT_YEAR: int = 10 # a lone digit ("1990-2") borrows the ref's decade _TWO_DIGIT_PIVOT: int = 50 # 2-digit years < 50 -> 2000s, >= 50 -> 1900s _TWO_DIGIT_YEAR: int = 100 # still < 100 after the pivot bump -> add the 1900s _MIN_FULL_YEAR: int = 1000 # below a plausible 4-digit year -> not a year _MODERN_CENTURY: int = 2000 _LEGACY_CENTURY: int = 1900 # Year parsing. An open-ended range end ("present"/"current", as in # "2002-present" or "2009, present") is not a year; consume it here so it does # not leak into the holder name. The range stays the start year. YEAR_RE: rex.Pattern[str] = rex.compile( r"\s*(?:[\s:([]*)?(?P\d{2,4})[]:\s]*" r"(?:(?:[-~=–—]|to)[\s:[]*(?:(?P\d{1,4})|present\b|current\b)[]:\s]*)?" # noqa: RUF001 r"[,/)]*(?:\s*(?:present|current)\b[\s,]*)?", rex.IGNORECASE, ) def _get_year(text: str, ref: int = 0) -> int | None: year = int(text) if year < _SINGLE_DIGIT_YEAR and ref: # Handle: 1990-2 fix = 0 if (ref % 10) < year else 1 year += ((ref // 10) + fix) * 10 if year < _TWO_DIGIT_PIVOT: year += _MODERN_CENTURY if year < _TWO_DIGIT_YEAR: year += _LEGACY_CENTURY if year < _MIN_FULL_YEAR: return None return year def _year_post_ignore(text: str, match: rex.Match[str]) -> bool: "Make sure we got a year" start = max(0, match.start("lo") - 1) end = min(len(text), match.end("lo") + 1) return bool( rex.match( r"[a-zA-Z]{0}|.{0}[a-zA-Z@]".format(match.group("lo")), text[start:end], ), ) def _split_by_years( text: str, years: YearRange | None = None, ) -> tuple[list[YearRange], list[str]]: "Split string by years found" years_list: list[YearRange] = [] if years is None else [years] parts: list[str] = [] if years is None else [""] last_end = 0 for year_match in YEAR_RE.finditer(text): if _year_post_ignore(text, year_match): continue match_dict = year_match.groupdict() low = _get_year(match_dict["lo"]) if low is None: continue high: int | None = low if match_dict.get("hi", None): high = _get_year(match_dict["hi"], low) part_len = year_match.start(0) - last_end if years is None or part_len >= MIN_PART_LEN: parts.append(text[last_end : year_match.start(0)]) years = YearRange(low=low, high=high) years_list.append(years) else: years.add(low) years.add(high) last_end = year_match.end(0) parts.append(text[last_end:]) return years_list, parts # Email parsing EMAIL_RE: rex.Pattern[str] = rex.compile( r"(?:^|[,;:\s<>/\\([])\s*(?P[^]),;:\s<>/\\([]+?@[^]),;:\s<>@/\\([]+?)\s*(?:[]),;:\s/\\<>]|$)", ) # "X, and Y" is two co-holders, not one. Split only on a comma-led "and" whose # right side starts a capitalised name, so lowercase tails ("and affiliates"), # already-trimmed prose (", and were"), and bare " and " (no comma) are left # alone. _COHOLDER_SPLIT_RE: rex.Pattern[str] = rex.compile(r",\s+and\s+(?=[A-Z])") def _split_coholders(name: str) -> list[str]: return [part for part in _COHOLDER_SPLIT_RE.split(name) if part.strip()] def _expand_coholders(pairs: list[tuple[str, str]]) -> list[tuple[str, str]]: """Split any emailless name that holds several "X, and Y" co-holders.""" expanded: list[tuple[str, str]] = [] for name, email in pairs: parts = [] if email else _split_coholders(name) if len(parts) > 1: expanded.extend((part, "") for part in parts) else: expanded.append((name, email)) return expanded def _get_holders_emails(text: str) -> list[tuple[str, str]]: holders_emails: list[tuple[str, str]] = [] # De-cruft email for sub in EMAIL_SUBS: text = sub.pattern.sub(sub.repl, text) last_end = 0 for email_match in EMAIL_RE.finditer(text): email = email_match.group("email") name = text[last_end : email_match.start()] last_end = email_match.end() holders_emails.append((name, email)) if len(text) - last_end >= MIN_PART_LEN: holders_emails.append((text[last_end:], "")) return _expand_coholders(holders_emails) # A holder name carrying code punctuation is a source fragment slurped after the # word "copyright", not a real name (emails are already split off, so <> is code). # Note: case is NOT a signal here - lowercase nicknames are valid holders; junk # lines that merely look lowercase are dropped via COPYRIGHT_*_IGNORE_REGEXES. _CODE_PUNCT_RE: rex.Pattern[str] = rex.compile(r"""[<>{}\[\]=;#"*|`\\]""") # A copyright line that runs into prose ends the name in a dangling clause like # ", and were [contributed by ...]". Trim a trailing comma-led clause whose verb # marks it as a sentence fragment - lowercase only, so org tails like ", Inc", # "and affiliates" or ", and Bar Corp" (a co-holder) are left intact. _TRAILING_PROSE_RE: rex.Pattern[str] = rex.compile( r",\s+(?:and\s+|or\s+)?(?:were|was|is|are|be|been|being)\b.*$", ) def _strip_trailing_prose(name: str) -> str: return _TRAILING_PROSE_RE.sub("", name).strip() def _looks_like_code_holder(name: str) -> bool: return bool(_CODE_PUNCT_RE.search(name)) def _clean_name(name: str | None) -> str: """Strip cruft and trailing prose from a raw holder name. Returns "" when nothing usable remains (a code fragment, or too short). """ if name is None: return "" name = name.strip(""",.;*'"@-–—[]{} \t""") # noqa: RUF001 for sub in NAME_CRUFT_SUBS: name = sub.pattern.sub(sub.repl, name) name = _strip_trailing_prose(name) if _looks_like_code_holder(name): return "" if len(name) < MIN_PART_LEN: return "" return name def _is_email_only(name: str, email: str, years: YearRange | None) -> bool: """The pair supplies only an email (no name, no years).""" return bool(not name and not years and email) def _is_name_only(name: str, email: str, years: YearRange | None) -> bool: """The pair supplies only a name (no email, no years).""" return bool(not email and not years and name) def _backfill_years( holders: list[CopyrightHolder], years: YearRange | None ) -> YearRange | None: """Fold a yearless pair's years into the previous holder if it has none. Returns the years to keep pending: None once merged, else unchanged. """ if years and holders and not holders[-1].years: holders[-1].years.merge(years) return None return years def _absorb_into_previous( last: CopyrightHolder, name: str, email: str, years: YearRange | None ) -> bool: """Complete the previous holder in place when this pair supplies only the piece it was missing (an email for a name, or a name for an email). Returns True when it did, so the caller does not start a new holder. """ if not last.email and _is_email_only(name, email, years): last.email = email return True if not last.name and _is_name_only(name, email, years): last.name = name return True return False def _process_name_email_pairs( name_email_pairs: list[tuple[str, str]], holders: list[CopyrightHolder], years: YearRange | None, ) -> YearRange | None: years_used = False for raw_name, email in name_email_pairs: name = _clean_name(raw_name) last = holders[-1] if holders else None # An empty pair only carries years: back-fill them onto the previous # holder rather than record a nameless, emailless entry. if not name and not email: years = _backfill_years(holders, years) continue # A continuation line that supplies only the piece the previous holder # was missing completes it in place instead of starting a new holder. if last is not None and _absorb_into_previous(last, name, email, years): continue holders.append(CopyrightHolder(name, email, years)) years_used = True return None if years_used else years def _get_holders_years( text: str, years: YearRange | None = None, ) -> abc.Iterator[tuple[YearRange, str]]: "Get pairs [(years, text)]" years_list, parts = _split_by_years(text, years) if len(parts[0]) < MIN_PART_LEN: return zip(years_list, parts[1:], strict=False) if len(parts[-1]) < MIN_PART_LEN: return zip(years_list, parts, strict=False) years_list = [YearRange(), *years_list] return zip(years_list, parts, strict=False) HOLDER_RE: rex.Pattern[str] = rex.compile( r'\s*(?:by\s*)?(?P\S.*?\S)[\s"\*,;/]*$', rex.IGNORECASE, ) def _advance_pending_years( pending: YearRange | None, pair_years: YearRange | None ) -> YearRange | None: """Fold this text-part's years into the years carried between parts. Merge when both are present; otherwise adopt the part's years when there are any (or nothing is pending yet); else leave the pending years unchanged. """ if pair_years and pending: pending.merge(pair_years) return pending if pending is None or pair_years: return pair_years return pending def _backfill_unmatched( holders: list[CopyrightHolder], pending: YearRange | None ) -> YearRange | None: """On a text part that yielded no holder, fold pending years into the previous holder when it lacks any; return the years still pending.""" if pending and holders and not holders[-1].years: holders[-1].years.merge(pending) return None return pending # Copyright holders factory def get_copyright_holders( copyright_: str, years: YearRange | None = None, ) -> tuple[list[CopyrightHolder], YearRange | None]: years_text_pairs = _get_holders_years(copyright_, years) holders: list[CopyrightHolder] = [] # To be merged or leftover to be returned pending_years: YearRange | None = None for pair_years, text in years_text_pairs: pending_years = _advance_pending_years(pending_years, pair_years) match = HOLDER_RE.match(text) if not match: # Discard current text, but keep what we found so far pending_years = _backfill_unmatched(holders, pending_years) continue name_email_pairs = _get_holders_emails(match.group("holder")) pending_years = _process_name_email_pairs( name_email_pairs, holders, pending_years, ) return holders, pending_years def _continuation_rest( text: str, prefix: str, alt_prefix: str, alt_text: str, years: YearRange | None ) -> str | None: """The holder text following a continuation prefix, or None to stop. Whitespace after the prefix marks a real continuation line and is consumed. With no such whitespace, only carry on when years are pending (a bare year wrapped onto its own line); otherwise there is nothing to continue. """ cont_match = rex.match(r"\s+", text[len(prefix) :]) or rex.match( r"\s+", alt_text[len(alt_prefix) :], ) if cont_match: return cont_match.string[cont_match.end() :] if years: return text[len(prefix) :] return None def _parse_copyright_continuation( text: str, continuation: rex.Match[str] | None, years: YearRange | None = None, ) -> tuple[str, rex.Match[str] | None]: if not continuation: return "", None lastindex = continuation.lastindex if continuation.lastindex is not None else 0 prefix = continuation.string[: continuation.start(lastindex)] # Special case for C style multiline comments; normalize tabs to spaces. alt_prefix = prefix.replace("/*", " *").replace("\t", " ") alt_text = text.replace("\t", " ") if not text.startswith(prefix) and not alt_text.startswith(alt_prefix): return "", None rest = _continuation_rest(text, prefix, alt_prefix, alt_text, years) # No continuation, or nothing but empty text left -> drop the continuation. if rest is None or rex.match(r"\s*$", rest): return "", None return rest, continuation def _re_split_iter( pattern: rex.Pattern[str], text: str, ) -> abc.Iterator[str]: end = 0 for indicator_match in pattern.finditer(text): part = text[end : indicator_match.start()] end = indicator_match.end() yield part yield text[end:] def _is_ignorable_notice(text: str, rest: str) -> bool: """Whether this line is not a real copyright notice. A pre-ignore pattern in the whole line (e.g. a #define, a po-file line) or a post-ignore pattern in the part after the indicator (mostly GPL boilerplate) disqualifies it. """ return any( ignore_pattern.search(text) for ignore_pattern in COPYRIGHT_PRE_IGNORE_RES ) or any( ignore_pattern.search(rest) for ignore_pattern in COPYRIGHT_POST_IGNORE_RES ) def _split_notice_parts(rest: str) -> list[str]: """Split the post-indicator text on any further indicators, de-cruft each part, and keep those still long enough to be a holder.""" copyrights: list[str] = [] for part_ in _re_split_iter(COPYRIGHT_INDICATOR_RE, rest): part = part_ if not part: continue for sub in CRUFT_SUBS: part = sub.pattern.sub(sub.repl, part) # len >= MIN_PART_LEN already implies a non-empty part. if len(part) >= MIN_PART_LEN: copyrights.append(part) return copyrights def parse_copyright( text: str, continuation: rex.Match[str] | None = None, years: YearRange | None = None, ) -> tuple[list[str] | None, rex.Match[str] | None]: if len(text) < MIN_PART_LEN or len(text) > MAX_NOTICE_LEN: return None, None match: rex.Match[str] | None = COPYRIGHT_INDICATOR_RE.search(text) if match: rest = match.string[match.end() :] else: # Process continuations rest, match = _parse_copyright_continuation(text, continuation, years) if not match: return None, None if _is_ignorable_notice(text, rest): return None, None return _split_notice_parts(rest), match def _add_new_holders( holders: list[CopyrightHolder], copyrights: list[str], years: YearRange | None, ) -> YearRange | None: # copyrights is a list of strings, that were split by copyright # indicators. for copyright_ in copyrights: new_holders, years = get_copyright_holders(copyright_, years) if new_holders: holders.extend(new_holders) elif years: # noqa: SIM102 # If we found a year, but not a holder, maybe the previous # holder is missing the years # Example: Copyright Maximiliano Curia (c) 2018 if len(holders) and not holders[-1].years: holders[-1].years.merge(years) years = None # The years might still be relevant for the next copyright holder. return years def iter_lines( text: str, start: int = 0, ) -> abc.Iterator[tuple[str, int, int]]: """Line iterator""" end = start - 1 while True: start = end + 1 end = text.find("\n", start) if end < 0: break yield text[start:end], start, end if text[start:]: yield text[start:], start, len(text) def parse_holders(content: str) -> list[CopyrightHolder]: """Find the copyright holders specified in content""" # list containing the copyrightHolders found holders: list[CopyrightHolder] = [] # last position checked last_seen = -1 # Iterate only for lines that might have a copyright indicator. for copyright_match in COPYRIGHT_PRE_INDICATOR_RE.finditer(content): # Already seen if copyright_match.start() < last_seen: continue match_start = copyright_match.start() line_start = content.rfind("\n", 0, match_start) + 1 # Initialize the parsing state (continuation and years) # continuation is set to the last copyright indicator found, that way we # can check the following lines, and correctly detect multiple copyright # holders after a single copyright indicator. Example: # Copyright: # Maximiliano Curia # Margarita Manterola continuation: rex.Match[str] | None = None # Keep years, to parse things like: # Copyright 2018 # Maximiliano Curia years: YearRange | None = None for line, _, last_seen_ in iter_lines(content, line_start): last_seen = last_seen_ copyrights, continuation = parse_copyright(line, continuation, years) # An empty continuation means that the copyright block ended. if not continuation: # remain calm and carry on in the outside iteration break if not copyrights: # A line without indicators, drop the cached years years = None continue # parse copyrights (list of strings) and create CopyrightHolders # with that information. Keep the years for the next iteration. years = _add_new_holders(holders, copyrights, years) return holders # Copyright holders aggregation _LEGAL_SUFFIX_RE = rex.compile( r",?\s+(?:Inc\.?|LLC|Ltd\.?|Corp\.?|GmbH|B\.V\.?|S\.A\.?|Co\.)$", rex.IGNORECASE, ) def _normalize_holder_name(name: str) -> str: """Return name lowercased with trailing legal-entity suffix stripped.""" return _LEGAL_SUFFIX_RE.sub("", name).strip().lower() def _emails_allow_merge(new: CopyrightHolder, previous: CopyrightHolder) -> bool: """Whether two same-named holders are the same entity. They merge unless both carry an email and the emails differ (two distinct people who happen to share a normalised name). """ return not (new.email and previous.email and new.email != previous.email) class CopyrightHolders: def __init__(self) -> None: self.by_normalized: dict[str, CopyrightHolder] = {} self.by_email: dict[str, CopyrightHolder] = {} def __len__(self) -> int: return len(self.by_normalized) def __iter__(self) -> abc.Iterator[CopyrightHolder]: return iter(self.by_normalized.values()) def __repr__(self) -> str: return f"CopyrightHolders({list(self.sorted_members())!s})" def add(self, copyright_: CopyrightHolder) -> CopyrightHolder: # Same email -> the same entity, merge onto the existing holder. if copyright_.email and copyright_.email in self.by_email: return self.by_email[copyright_.email].merge(copyright_) normalized = _normalize_holder_name(copyright_.name) previous = self.by_normalized.get(normalized) if previous is not None and _emails_allow_merge(copyright_, previous): # Same name, compatible emails: merge, adopting the new email when # the existing holder had none. if copyright_.email and not previous.email: self.by_email[copyright_.email] = previous return previous.merge(copyright_) # A new entry. A free name keys by its normalised form; a name that # collides with a different-email entity keys by the full person, so # both survive side by side. key = copyright_.person.lower() if previous is not None else normalized if copyright_.email: self.by_email[copyright_.email] = copyright_ self.by_normalized[key] = copyright_ return copyright_ def extend(self, copyrights: abc.Iterable[CopyrightHolder]) -> None: for copyright_ in copyrights: self.add(copyright_) def key(self) -> tuple[str, ...]: return tuple( sorted( copyright_.email or copyright_.person for copyright_ in self.by_normalized.values() ), ) def merge(self, other: CopyrightHolders) -> CopyrightHolders: for value in other.by_normalized.values(): self.add(value) return self def sorted_members(self) -> abc.Iterator[str]: return ( str(self.by_normalized[key]) for key in sorted(self.by_normalized.keys()) ) decopy-0.3.2/decopy/licensing/license_expr.py000066400000000000000000000335071522176660400213100ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Licence-expression AST, parsers and renderers. A single AST (License/With/And/Or) is the internal representation for licence expressions. SPDX expressions parse into it via parse_spdx_expression(); it renders to DEP-5 short-name form via render_dep5_expression(). The leaf conversions (SPDX id -> DEP-5 name, SPDX exception id -> DEP-5 label) are spdx-specific knowledge, so they are injected into the renderer rather than imported -- this module is a leaf and never imports spdx. """ from __future__ import annotations import re as reo import typing as t from collections.abc import Callable, Iterator class License(t.NamedTuple): id: str class With(t.NamedTuple): license: License exception: str class And(t.NamedTuple): terms: tuple class Or(t.NamedTuple): terms: tuple type Node = License | With | And | Or # SPDX licence-expression parser # # Grammar (precedence, tightest -> loosest: WITH > AND > OR; parens override): # or-expr := and-expr ( "OR" and-expr )* # and-expr := with-expr ( "AND" with-expr )* # with-expr := atom ( "WITH" exception-id )? # atom := license-id | "(" or-expr ")" # AND/OR/WITH are case-sensitive uppercase operators; a license/exception id is # any whitespace-free run that is not a parenthesis. The SPDX "+" operator is # always attached to its id (e.g. GPL-2.0+), so it rides along inside the id # token and is resolved by the leaf converter -- no separate handling needed. _TOKEN_RE = reo.compile(r"\(|\)|\bAND\b|\bOR\b|\bWITH\b|[^\s()]+") _OPERATORS = frozenset({"(", ")", "AND", "OR", "WITH"}) class _ExprParser: """Shared token cursor for the recursive-descent expression parsers.""" _kind = "expression" def __init__(self, text: str, tokens: list[str]) -> None: self.text = text self.tokens = tokens self.pos = 0 def _fail(self, reason: str) -> t.NoReturn: msg = f"{reason} in {self._kind}: {self.text!r}" raise ValueError(msg) def _peek(self) -> str | None: return self.tokens[self.pos] if self.pos < len(self.tokens) else None def _advance(self) -> str: if self.pos >= len(self.tokens): self._fail("unexpected end") tok = self.tokens[self.pos] self.pos += 1 return tok class _SpdxParser(_ExprParser): """Recursive-descent parser for an SPDX licence expression.""" _kind = "SPDX expression" def parse(self) -> Node: tree = self._or() if self.pos != len(self.tokens): self._fail("trailing tokens") return tree def _or(self) -> Node: terms = [self._and()] while self._peek() == "OR": self._advance() terms.append(self._and()) return terms[0] if len(terms) == 1 else Or(tuple(terms)) def _and(self) -> Node: terms = [self._with()] while self._peek() == "AND": self._advance() terms.append(self._with()) return terms[0] if len(terms) == 1 else And(tuple(terms)) def _with(self) -> Node: node = self._atom() if self._peek() != "WITH": return node self._advance() # SPDX: WITH applies to a licence id only, not a parenthesised group. if not isinstance(node, License): self._fail("WITH must follow a licence id") exc = self._advance() if exc in _OPERATORS: self._fail("expected exception id after WITH") return With(node, exc) def _atom(self) -> Node: tok = self._peek() if tok is None: self._fail("unexpected end") if tok == "(": self._advance() node = self._or() if self._peek() != ")": self._fail("expected ')'") self._advance() return node if tok in _OPERATORS: self._fail(f"unexpected {tok!r}") self._advance() return License(tok) def _normalize_operator(token: str) -> str: """Uppercase a token that is an operator word, leaving ids untouched. SPDX mandates uppercase AND/OR/WITH, but real-world source writes them in any case (e.g. `MIT or Apache-2.0`). The detection regex already matches operators case-insensitively, so accept them here too. No SPDX id or exception id is the bare word and/or/with, so this cannot corrupt an id; parens are separate tokens and are left alone. """ upper = token.upper() return upper if upper in {"AND", "OR", "WITH"} else token def parse_spdx_expression(text: str) -> Node: """Parse a full SPDX licence expression into an AST. Operators are accepted case-insensitively. Raises ValueError on a malformed or incomplete expression (unbalanced parens, dangling operator, trailing tokens, empty input). """ tokens = [_normalize_operator(tok) for tok in _TOKEN_RE.findall(text)] return _SpdxParser(text, tokens).parse() # DEP-5 licence-expression parser (copyright-format 1.0 sec 7.2) # # Grammar (loosest -> tightest); the comma is a precedence level looser than # `or`, applied as a left-associative fold: # comma-expr := or-expr ( "," ("and"|"or") or-expr )* # or-expr := and-expr ( "or" and-expr )* # and-expr := with-expr ( "and" with-expr )* # with-expr := license-id ( "with" label "exception" )? # Operators are lowercase, matched case-insensitively; tokens split on # whitespace and commas, so a licence name merely containing "and"/"or" (e.g. # "Sandia") is never split. The reader is lenient: it accepts the documented # comma-before-`and` override and the undocumented `, or` / multi-comma forms. _DEP5_TOKEN_RE = reo.compile(r",|[^\s,]+") _DEP5_KEYWORDS = frozenset({"and", "or", "with", "exception"}) class _Dep5Parser(_ExprParser): """Recursive-descent parser for a DEP-5 short-name licence expression.""" _kind = "DEP-5 expression" def parse(self) -> Node: tree = self._comma_expr() if self.pos != len(self.tokens): self._fail("trailing tokens") return tree def _peek_is(self, word: str) -> bool: tok = self._peek() return tok is not None and tok.lower() == word def _comma_expr(self) -> Node: acc = self._or_expr() while self._peek() == ",": self._advance() op = self._advance() if op.lower() not in ("and", "or"): self._fail("expected 'and' or 'or' after comma") rhs = self._or_expr() acc = And((acc, rhs)) if op.lower() == "and" else Or((acc, rhs)) return acc def _or_expr(self) -> Node: terms = [self._and_expr()] while self._peek_is("or"): self._advance() terms.append(self._and_expr()) return terms[0] if len(terms) == 1 else Or(tuple(terms)) def _and_expr(self) -> Node: terms = [self._with_expr()] while self._peek_is("and"): self._advance() terms.append(self._with_expr()) return terms[0] if len(terms) == 1 else And(tuple(terms)) def _with_expr(self) -> Node: node = self._atom() if not self._peek_is("with"): return node self._advance() # "with" label = self._advance() if label.lower() in _DEP5_KEYWORDS: self._fail("expected exception label after 'with'") if not self._peek_is("exception"): self._fail("expected 'exception' to close 'with' clause") self._advance() # "exception" return With(node, label) def _atom(self) -> License: tok = self._peek() if tok is None: self._fail("unexpected end") if tok == "," or tok.lower() in _DEP5_KEYWORDS: self._fail(f"unexpected {tok!r}") self._advance() return License(tok) def parse_dep5_expression(text: str) -> Node: """Parse a DEP-5 short-name licence expression into an AST. Raises ValueError on empty or malformed input. The reader is lenient (see the grammar comment): it parses the documented comma-before-`and` override and the undocumented `, or` / multi-comma forms a real-world field may carry. """ return _Dep5Parser(text, _DEP5_TOKEN_RE.findall(text)).parse() class UnconvertibleError(Exception): """An expression cannot be fully rendered to DEP-5 (e.g. unknown exception).""" def _identity(name: str) -> str: return name def _render_with( node: With, convert_id: Callable[[str], str], convert_exception: Callable[[str], str | None], ) -> str: base = convert_id(node.license.id) label = convert_exception(node.exception) if label is None: raise UnconvertibleError return f"{base} with {label} exception" def render_dep5_expression( node: Node, convert_id: Callable[[str], str] = _identity, convert_exception: Callable[[str], str | None] = _identity, ) -> str: """Render a licence-expression AST to its DEP-5 short-name form. convert_id maps a leaf id to its DEP-5 name and convert_exception maps an exception id to its DEP-5 label (returning None when unknown, which raises UnconvertibleError). Both default to identity, which is what a DEP-5-sourced AST -- whose leaves already hold DEP-5 names/labels -- round-trips through; the SPDX path injects the spdx.py converters. """ if isinstance(node, License): return convert_id(node.id) if isinstance(node, With): return _render_with(node, convert_id, convert_exception) if isinstance(node, And): # DEP-5 sec 7.2: `and` binds tighter than `or`, so an `or` operand of an # `and` must be set off with a comma (", and") to bind first. Plain # " and " otherwise. (Mixed nesting deeper than one comma level cannot # be expressed; such rare trees are rendered best-effort.) parts = [ render_dep5_expression(term, convert_id, convert_exception) for term in node.terms ] sep = ", and " if any(isinstance(term, Or) for term in node.terms) else " and " return sep.join(parts) return " or ".join( render_dep5_expression(term, convert_id, convert_exception) for term in node.terms ) def _spdx_paren( node: Node, convert_name: Callable[[str], str], convert_exception: Callable[[str], str], ) -> str: """Render `node` as an SPDX operand, parenthesising a compound (And/Or).""" inner = render_spdx_expression(node, convert_name, convert_exception) return f"({inner})" if isinstance(node, (And, Or)) else inner def render_spdx_expression( node: Node, convert_name: Callable[[str], str] = _identity, convert_exception: Callable[[str], str] = _identity, ) -> str: """Render a licence-expression AST to an unambiguous SPDX string. Every compound operand is parenthesised, so the result round-trips through parse_spdx_expression(). Used both as the fallback for trees DEP-5 cannot express and as the --spdx output path. convert_name maps a leaf's DEP-5 name to its SPDX id and convert_exception maps an exception label to its SPDX id (both default to identity, which leaves an SPDX-sourced AST unchanged). """ if isinstance(node, License): return convert_name(node.id) if isinstance(node, With): base = convert_name(node.license.id) return f"{base} WITH {convert_exception(node.exception)}" joiner = " AND " if isinstance(node, And) else " OR " return joiner.join( _spdx_paren(term, convert_name, convert_exception) for term in node.terms ) def map_leaves( node: Node, convert_id: Callable[[str], str] = _identity, convert_exception: Callable[[str], str] = _identity, ) -> Node: """Rebuild `node` with each leaf name (and With exception) converted. Structural only: moves an AST between vocabularies (e.g. an SPDX-parsed tree into DEP-5 names) without touching its and/or/with shape. """ if isinstance(node, License): return License(convert_id(node.id)) if isinstance(node, With): return With( License(convert_id(node.license.id)), convert_exception(node.exception) ) mapped = (map_leaves(term, convert_id, convert_exception) for term in node.terms) if isinstance(node, And): return And(tuple(mapped)) return Or(tuple(mapped)) def _leaf_keys(node: Node) -> Iterator[str]: """Yield each leaf's DEP-5 key (a With's full 'NAME with LABEL exception').""" if isinstance(node, License): yield node.id elif isinstance(node, With): yield render_dep5_expression(node) else: for term in node.terms: yield from _leaf_keys(term) def license_atoms(node: Node) -> list[str]: """The leaf licence names of an expression, in encounter order, de-duplicated. A With node contributes its full DEP-5 'NAME with LABEL exception' string, which is the licence key decopy groups by. Used by the read path to populate a group's licence set from a parsed expression regardless of its and/or shape. """ atoms: list[str] = [] seen: set[str] = set() for key in _leaf_keys(node): if key not in seen: seen.add(key) atoms.append(key) return atoms decopy-0.3.2/decopy/licensing/license_parsers.py000066400000000000000000000336601522176660400220110ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2018, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """License detail parsers and the SUBS evaluation helpers. The get_detail callbacks and parse_* family parsers that refine a matched license id, plus the shared SUBS helpers. Consumes the data in license_subs; wired onto the rules in res via get_detail / _FAMILY_DETAIL. """ from __future__ import annotations import collections.abc as abc import itertools import re as reo from decopy.common._regex import rex from decopy.common.re_types import ReSub, ReSubGroup from decopy.licensing.license_subs import ( APACHE_EXCEPTION_SUBS, ARTISTIC_SUBS, ASWF_SUBS, BOEHM_SUBS, BSD_SUBS, CC_BY_SUBS, CECILL_SUBS, CMU_MACH_SUBS, CNRI_SUBS, CTAN_SUBS, DOC_SUBS, EFL_SUBS, ESA_PL_SUBS, FONT_SUBS, GFDL_SUBS, GNU_EXCEPTION_SUBS, GNU_VERSION_SUBS, HW_SUBS, LPPL_SUBS, MANPAGE_SUBS, MIT_SUBS, MITRE_SUBS, MULAN_SUBS, OSL_DERIVED_SUBS, PD_MISC_SUBS, QPL_EXCEPTION_SUBS, SRCAVAIL_SUBS, SSH_SUBS, STD_SUBS, TU_BERLIN_SUBS, UDA_SUBS, UNIVERSAL_EXCEPTION_SUBS, USGOV_PD_SUBS, ZPL_SUBS, ) # Match type accepted by the family detail callbacks. They ignore the match # argument, so it also admits None (callers may pass it explicitly). type DetailMatch = rex.Match[str] | reo.Match[str] | None # LICENSES_RES common get_details def name_version( text: str, version_match: rex.Match[str] | reo.Match[str], group: str, name: str | None = None, ) -> str: _ = text if not name: name = group version = version_match.group(1) return "{}{}".format(name, "-{}".format(version.rstrip(".0") if version else "")) def group_version( text: str, version_match: rex.Match[str] | reo.Match[str], group: str, ) -> str: return name_version(text, version_match, group) def python_detail( text: str, _match: DetailMatch, _group: str, ) -> str: # The Python license text comes in three shapes: the standalone PSF license # (PSF-2.0) and the 2.0 / 2.0.1 collections that wrap PSF + BeOpen + CNRI + # CWI. The collection is marked by the BeOpen heading; 2.0.1 is the variant # whose CNRI agreement is the GPL-compatible "for Python" form. if "CNRI LICENSE AGREEMENT FOR PYTHON" in text: return "Python-2.0.1" if "BEOPEN PYTHON OPEN SOURCE LICENSE" in text: return "Python-2" return "PSF-2" def cc_at_detail( text: str, _match: DetailMatch, _group: str, ) -> str: # The two Austrian CC 3.0 legal codes have no deed title for the generic # CC_BY_SUBS path to read (CC-BY-3.0-AT has no version string in its text at # all), so emit the full id directly. They are matched on "Österreich", which # is unique to these two files; the ShareAlike clause tells them apart. if "Weitergabe unter gleichen Bedingungen" in text: return "CC-BY-SA-3-AT" return "CC-BY-3-AT" def caldera_detail( text: str, _match: DetailMatch, _group: str, ) -> str: # Caldera and Caldera-no-preamble share the same BSD-4-clause body; the full # Caldera licence additionally carries the ancient-UNIX "fee free license" # preamble. One family rule (anchored on the shared body) tells them apart. if "hereby grants a fee free license" in text: return "Caldera" return "Caldera-no-preamble" def ijg_detail( text: str, _match: DetailMatch, _group: str, ) -> str: # IJG and IJG-short share the README / no-warranty permission clause; the full # IJG licence additionally carries the "derived from or based on the IJG code" # condition that the short form drops. if "derived from or based on the IJG code" in text: return "IJG" return "IJG-short" def libpng_detail( text: str, version_match: rex.Match[str] | reo.Match[str], _group: str, ) -> str: # The PNG Reference Library License title carries the major version: v1 is the # historic libpng (0.5 through 1.6.35), v2 the current form. _ = text return "libpng-1.6.35" if version_match.group(1) == "1" else "libpng-2" def mackerras_detail( text: str, _match: DetailMatch, _group: str, ) -> str: # Both Mackerras BSD-3-clause variants share the disclaimer; the acknowledgment # form adds a "developed by Paul Mackerras" attribution clause. if "developed by Paul Mackerras" in text: return "Mackerras-3-Clause-acknowledgment" return "Mackerras-3-Clause" # subs are way more efficient when using python's re (reo here) # SUBS evaluation helpers, shared by the get_detail callbacks below and by # find_licenses (which applies a matched rule's exception_subs). def _strip_zero_version(value: str) -> str: # Drop a trailing decimal ".0" (e.g. "GFDL-1.0" -> "GFDL-1") without eating a # bare trailing zero that belongs to the name itself (e.g. "MIT-0"). return rex.sub(r"\.0*$", "", value) def _parse_subs_first( text: str, fallback: str, subs: abc.Iterable[ReSub], ) -> str: for sub in subs: match = sub.pattern.search(text) if not match: continue return _strip_zero_version(match.expand(sub.repl)) return fallback def _parse_subs_all( text: str, subs: abc.Iterable[ReSub | ReSubGroup], ) -> list[str]: found: dict[str, None] = {} def _try(sub: ReSub) -> bool: match = sub.pattern.search(text) if match: found[_strip_zero_version(match.expand(sub.repl))] = None return bool(match) for item in subs: if isinstance(item, ReSubGroup): any(_try(sub) for sub in item.subs) else: _try(item) return list(found) def parse_exceptions(text: str, group: str, exception_subs: abc.Iterable) -> str: exceptions = _parse_subs_all( text, itertools.chain(exception_subs, UNIVERSAL_EXCEPTION_SUBS) ) exception = ( " with {} exception".format("_".join(sorted(exceptions))) if exceptions else "" ) return group + exception # Per-family get_detail callbacks: refine a matched family id using its SUBS. # Wired onto the matching rules below via _FAMILY_DETAIL. def parse_artistic(text: str, _match: DetailMatch, group: str) -> str: return _parse_subs_first(text, group, ARTISTIC_SUBS) def parse_bsd(text: str, _match: DetailMatch, group: str) -> str: # A bare BSD warranty disclaimer is not enough to assert BSD: only report it # when a specific clause count or variant is recognised. Otherwise return "" # so a disclaimer borrowed by an ISC/MIT-style licence is not mislabelled BSD # (find_licenses drops empty results). return _parse_subs_first(text, "", BSD_SUBS) def parse_cc_by(text: str, _match: DetailMatch, group: str) -> str: details = _parse_subs_all(text, CC_BY_SUBS) return group + "".join(details) def parse_doc(text: str, _match: DetailMatch, group: str) -> str: # The DocLicense gate fires on a title alone; require a body clause here so a # mere reference to one of these licences is not reported as the licence. return _parse_subs_first(text, "", DOC_SUBS) def parse_font(text: str, _match: DetailMatch, group: str) -> str: # The FontLicense gate groups the font licences; the per-licence rule must # match a substantive grant clause, not just a foundry or font name. return _parse_subs_first(text, "", FONT_SUBS) def parse_hw(text: str, _match: DetailMatch, group: str) -> str: # The HardwareLicense gate groups the open-hardware licences; each rule pairs # the versioned title with a preamble clause so references do not match. return _parse_subs_first(text, "", HW_SUBS) def parse_usgov_pd(text: str, _match: DetailMatch, group: str) -> str: # The USGovPD gate groups the US-Government / agency public-domain notices; # each rule pins the public-domain clause to the agency naming the id. return _parse_subs_first(text, "", USGOV_PD_SUBS) def parse_pd_misc(text: str, _match: DetailMatch, group: str) -> str: # The PdMisc gate groups unrelated public-domain / "do as you wish" notices; # each rule anchors on the notice's distinctive dedication clause. return _parse_subs_first(text, "", PD_MISC_SUBS) def parse_manpage(text: str, _match: DetailMatch, group: str) -> str: # The ManPage gate groups the Linux man-pages licences; specific variant # tails come first, the bare two-paragraph form is the fallback. return _parse_subs_first(text, "", MANPAGE_SUBS) def parse_ctan(text: str, _match: DetailMatch, group: str) -> str: # The CtanTex gate groups the CTAN/TeX licences; each rule anchors on its # distinctive permission/rename clause. return _parse_subs_first(text, "", CTAN_SUBS) def parse_std(text: str, _match: DetailMatch, group: str) -> str: # The StandardsBody gate groups Unicode + the W3C-derived licences; the ICU # Unicode rule precedes the bare-title 2015 fallback. return _parse_subs_first(text, "", STD_SUBS) def parse_uda(text: str, _match: DetailMatch, group: str) -> str: # The UdaData gate groups the Microsoft Use of Data Agreements; each rule # anchors on the Computational/Open title with its version. return _parse_subs_first(text, "", UDA_SUBS) def parse_aswf(text: str, _match: DetailMatch, group: str) -> str: # The AswfAssets gate groups the ASWF Digital Assets versions; each rule # anchors on the title version line. return _parse_subs_first(text, "", ASWF_SUBS) def parse_mitre(text: str, _match: DetailMatch, group: str) -> str: # The MitreTou gate groups the MITRE terms-of-use grants; each rule names the # dataset (CAPEC, CVE). return _parse_subs_first(text, "", MITRE_SUBS) def parse_cmu_mach(text: str, _match: DetailMatch, group: str) -> str: # The CmuMach gate groups the CMU/Mach permission notices; the documentation # variant precedes the bare no-doc fallback. return _parse_subs_first(text, "", CMU_MACH_SUBS) def parse_ssh(text: str, _match: DetailMatch, group: str) -> str: # The Ssh gate groups Tatu Ylonen's SSH notices; the full OpenSSH form # precedes the short grant-only fallback. return _parse_subs_first(text, "", SSH_SUBS) def parse_boehm(text: str, _match: DetailMatch, group: str) -> str: # The BoehmGc gate groups the Boehm GC permission notices; the without-fee # variant precedes the base form. return _parse_subs_first(text, "", BOEHM_SUBS) def parse_mulan(text: str, _match: DetailMatch, group: str) -> str: # The MulanPsl gate groups the two Mulan PSL versions; each rule anchors on # the "第1版"/"第2版" version marker inside the grant clause. return _parse_subs_first(text, "", MULAN_SUBS) def parse_osl_derived(text: str, _match: DetailMatch, group: str) -> str: # The OslDerived gate groups the OSL-template derivatives (TGPPL, UCL); each # rule pairs the licence's own title with the shared OSL body clause. return _parse_subs_first(text, "", OSL_DERIVED_SUBS) def parse_esa_pl(text: str, _match: DetailMatch, group: str) -> str: # The EsaPl gate groups the three ESA-PL copyleft strengths; each rule # anchors on its variant title plus the start of the shared definitions. return _parse_subs_first(text, "", ESA_PL_SUBS) def parse_tu_berlin(text: str, _match: DetailMatch, group: str) -> str: # The TuBerlin gate groups the two TU-Berlin notice versions; the v2.0 rule # (appended grant) precedes the v1.0 base-notice fallback. return _parse_subs_first(text, "", TU_BERLIN_SUBS) def parse_srcavail(text: str, _match: DetailMatch, group: str) -> str: # The SrcAvailable gate groups the source-available / business copyleft # licences; each rule pairs the title-and-version line with a substantive # body clause. The two Parity rules precede nothing else they could shadow. return _parse_subs_first(text, "", SRCAVAIL_SUBS) def parse_cecill(text: str, _match: DetailMatch, group: str) -> str: return _parse_subs_first(text, group, CECILL_SUBS) def parse_cnri(text: str, _match: DetailMatch, group: str) -> str: return _parse_subs_first(text, group, CNRI_SUBS) def parse_efl(text: str, _match: DetailMatch, group: str) -> str: return _parse_subs_first(text, group, EFL_SUBS) def parse_gpl(text: str, _match: DetailMatch, group: str) -> str: version = _parse_subs_first(text, "", GNU_VERSION_SUBS) base = "{}{}".format( group, "-{}".format(version.rstrip("0.")) if version else "", ) return parse_exceptions(text, base, GNU_EXCEPTION_SUBS) def parse_gfdl(text: str, _match: DetailMatch, group: str) -> str: details = _parse_subs_all(text, GFDL_SUBS) return group + "".join(details) def parse_lppl(text: str, _match: DetailMatch, group: str) -> str: details = _parse_subs_all(text, LPPL_SUBS) return group + "".join(details) def parse_mit(text: str, _match: DetailMatch, group: str) -> str: return _parse_subs_first(text, group, MIT_SUBS) def parse_apache(text: str, _match: DetailMatch, group: str) -> str: return parse_exceptions(text, group, APACHE_EXCEPTION_SUBS) def parse_qpl(text: str, _match: DetailMatch, group: str) -> str: return parse_exceptions(text, group, QPL_EXCEPTION_SUBS) def parse_zpl(text: str, _match: DetailMatch, group: str) -> str: details = _parse_subs_all(text, ZPL_SUBS) return group + "".join(details) decopy-0.3.2/decopy/licensing/license_prefilter.py000066400000000000000000000150341522176660400223210ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Literal prefilter for license detection. find_licenses() runs up to 512 regex searches per file; on real source almost all fail (see the perf analysis). Every rule has a literal that MUST appear in any match, so for a given file only the rules whose literal is present need to run. This module extracts those necessary literals from the compiled rule patterns and exposes candidate_indices(text): the set of rule indices worth running. Optional dependency: the multi-literal scan uses pyahocorasick. Without it, candidate_indices() returns None and the caller runs every rule - same result, just slower. The extractor uses re's internal parser (re._parser); if that ever changes, extraction degrades to "always run that rule", never to a wrong answer. """ from __future__ import annotations import contextlib from typing import Any from decopy.licensing.res import LICENSES_RES try: # internal API; a per-rule failure just marks the rule always-run (below) import re._parser as _sre # type: ignore[import-not-found] # ty: ignore[unresolved-import] except ImportError: # pragma: no cover - re._parser has existed since 3.11 _sre = None # type: ignore[assignment] try: import ahocorasick # type: ignore[import-not-found] # ty: ignore[unresolved-import] except ImportError: ahocorasick = None # type: ignore[assignment] # Shortest literal we trust as a prefilter anchor. All current rules have a # necessary literal well above this; a shorter one just makes the rule always-run. _MIN_LITERAL_LEN = 4 def _lit_runs(subpattern: Any) -> list[str]: # noqa: ANN401 """Runs of consecutively-required literals (casefolded), in order. Only unconditionally-required LITERALs contribute; an optional repeat, char-class, ANY or alternation breaks the run. Conservative by construction: it can shorten or drop a literal, never invent one that could be absent. """ runs: list[str] = [] cur: list[str] = [] def flush() -> None: if cur: runs.append("".join(cur)) cur.clear() for op, av in subpattern: if op.name == "LITERAL": cur.append(chr(av).casefold()) continue flush() if op.name == "SUBPATTERN": runs.extend(_lit_runs(av[-1])) elif op.name in ("MAX_REPEAT", "MIN_REPEAT") and av[0] >= 1: runs.extend(_lit_runs(av[2])) flush() return runs def _rule_literals(pattern: str, flags: int) -> list[str] | None: """Literals where at least one MUST be present for the rule to match. Returns None (-> the rule must always run) when no usable necessary literal exists: a top-level alternation whose every branch lacks one, or a sequence whose longest required literal is too short. """ if _sre is None: return None parsed = _sre.parse(pattern, flags) data = list(parsed) if len(data) == 1 and data[0][0].name == "BRANCH": branch_lits = [ max(_lit_runs(branch), key=len, default="") for branch in data[0][1][1] ] if all(len(lit) >= _MIN_LITERAL_LEN for lit in branch_lits): return branch_lits return None longest = max(_lit_runs(parsed), key=len, default="") return [longest] if len(longest) >= _MIN_LITERAL_LEN else None class _LiteralPrefilter: """Each rule's necessary literal(s) -> its index, via one Aho-Corasick automaton, so candidate_indices() can name the rules worth running. Inactive (candidate_indices -> None) when pyahocorasick is absent or no rule yields a usable literal; the caller then runs every rule. """ def __init__(self) -> None: always_run, lit_to_rules = self._extract_literals() self._always_run = always_run self._automaton = self._compile(lit_to_rules) @staticmethod def _extract_literals() -> tuple[frozenset[int], dict[str, set[int]]]: """Per rule, the literals that select it; or mark it always-run.""" always_run: set[int] = set() lit_to_rules: dict[str, set[int]] = {} for index, rule in enumerate(LICENSES_RES): literals: list[str] | None = None with contextlib.suppress(Exception): literals = _rule_literals(rule.re, int(rule.re_flags)) if literals is None: always_run.add(index) continue for literal in literals: lit_to_rules.setdefault(literal, set()).add(index) return frozenset(always_run), lit_to_rules @staticmethod def _compile(lit_to_rules: dict[str, set[int]]) -> Any: # noqa: ANN401 """The Aho-Corasick automaton over the literals, or None if unavailable.""" if ahocorasick is None or not lit_to_rules: return None automaton = ahocorasick.Automaton() for literal, indices in lit_to_rules.items(): automaton.add_word(literal, frozenset(indices)) automaton.make_automaton() return automaton @property def available(self) -> bool: return self._automaton is not None def candidate_indices(self, text: str) -> frozenset[int] | None: if self._automaton is None: return None candidates = set(self._always_run) for _end, indices in self._automaton.iter(text.casefold()): candidates |= indices return frozenset(candidates) _PREFILTER = _LiteralPrefilter() def prefilter_available() -> bool: """True when the literal prefilter is active (pyahocorasick present).""" return _PREFILTER.available def candidate_indices(text: str) -> frozenset[int] | None: """Rule indices worth running for `text`: those whose necessary literal is present, plus the always-run rules. None -> prefilter unavailable, run all. """ return _PREFILTER.candidate_indices(text) decopy-0.3.2/decopy/licensing/license_subs.py000066400000000000000000004023761522176660400213120ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2018, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Regex substitution tables and copyright-detection patterns. Leaf data module: the SUBS tables consumed by the license detail parsers and find_licenses, plus the copyright-notice regexes used by holders.py. Contains no logic; the parsers live in license_parsers, the rules in res. """ from __future__ import annotations import re as reo from decopy.common._regex import rex from decopy.common.re_types import ReFlags, ReSub, ReSubGroup COMMENTS_SUBS: tuple[ReSub, ...] = ( ReSub( reo.compile( # Fortran (fortran is weird: https://gcc.gnu.org/ml/gcc/2000-01/msg00250.html) r"^C[ \t]" # C / C++ (avoid dropping :// from uris) r"|(?\+-]?="), ReFlags(r"\(c\)\ ?\)"), ReFlags(r"sizeof\([^)]+\)"), ReFlags(r"\(c\)(?:->|\.)[^ ]"), ReFlags(r"=\s*\(c\)\s*\*.*;"), ReFlags(r"\(c\)\s*[*/+-]\s*\("), ReFlags(r"\(c\)\ and\ its\ data"), ReFlags(r"character\ \(c\)"), ReFlags(r"\(c\)\ differs\ for"), ReFlags(r"combining\ mark\ \(c\)"), ReFlags(r"\(c\)\ *processed"), ReFlags(r"\(c\)\ data\ (?:[^ ]+\ ){,5}has\ been\ received"), ReFlags(r"shown\sby\s\(c\)\ssupports"), ReFlags(r"AND\s\(c\)"), # more text ReFlags(r"following\scopyright\sand\slicense "), ReFlags(r"\bcopyright\ssign\b"), # found in qca ReFlags(r'"Country\ Code\ \(C\)"'), # found in cinnamon ReFlags(r"\bcopyright\ and\b"), # template used in po ReFlags(r"\bCopyright:?\s(?:\(\s*C\s*\)\s)?YEAR\b"), # found in cinnamon-settings-daemon ReFlags(r"free\sof\sknown\scopyright\srestrictions"), ReFlags(r"[-\s–—]%[ds][-\s–—]"), # noqa: RUF001 # template (marble) ReFlags(r"<\*\*authorname\*\*>"), # template (kcoreaddons) ReFlags(r"%{AUTHOR}"), # found in kcoreaddons ReFlags( r'i18n\("Copyright (?:[^ ]+ ){,4}' r"(?:Developer|Foo Foundation|Bar Foundation|Baz Foundation)", ), # tagged to be ignored ReFlags(r"krazy:exclude(?:all)?=[^\s]*copyright"), # found in kde-l10n ReFlags(r"copyright\ of\ every\ file\ belongs\ to\ its\s"), # Test case in ark ReFlags( r"UNRAR\s+[0-9.]+\ (\(iconv\)|beta\ [0-9])\ freeware\s+" r"Copyright\ \(c\)\ [0-9-]+\ Alexander\ Roshal", ), # dll files interfaces ReFlags(r"\{[0-9A-F-]{32,}\}"), ) COPYRIGHT_PRE_IGNORE_RES: tuple[rex.Pattern[str], ...] = tuple( rex.compile(re_flag.re, re_flag.flags) for re_flag in COPYRIGHT_PRE_IGNORE_REGEXES ) COPYRIGHT_INDICATOR_REGEXES: tuple[ReFlags, ...] = ( # Multiline indicator ReFlags( r"(?:^|\s|\W)" r"(" r"(?:[^ ]+\ ){,8}(?:is|are)\ " r"(?:" r"copyright\ by\ the[\s]+" r"|Copyright(?:\ \(C\))?[:\s]+" r"|the\ Copyright\ (?:<\w+>)?property\ of(?:[:\s]+|$)" r")" r")" r"(?=\S|$)", ), # The full word ReFlags(r"(?:^|\s|\W)(copyright(?:\s*\(c\))?(?:[:\s]+))(?=\S|$)"), # Legally-valid abbreviation ReFlags(r"(?:^|\s|\W)(copr\.(?:[:\s]+))(?=\S|$)"), # Unicode character COPYRIGHT SIGN ReFlags(r"(?:^|\s)(©(?:[:\s]+))(?=\S|$)"), # Unicode copyright sign encoded in iso8859 ReFlags(r"(?:^|\s)(\xa9(?:[:\s]+))(?=\S|$)"), # HTML copyright sign ReFlags(r"(?:^|\s|\W)(©(?:[:\s]+))(?=\S|$)"), # texi macro ReFlags(r"(?:^|\s|\W)(@copyright\{?\}?(?:[:\s]+))(?=\S|$)"), # html copyright sign ReFlags(r"(?:^|\s|\W)(&\#169;(?:[:\s]+))(?=\S|$)"), # Free text found in some debian/copyright ReFlags( r"(?:^|\s|\W)((?:Upstream\ Authors?\ ?(?:and|,)\ )?Copyright\ Holders(?:[:\s]+))(?=\S|$)", # noqa: E501 ), # Legally-null representation of sign ReFlags(r"(?:^|\s|\W)((?:\(\s?c\s?\))[:\s]*)(?=\S|$)"), ReFlags(r"(?:^|\s|\W)(SPDX-FileCopyrightText(?:\s*\(c\))?(?:[:\s]+))(?=\S|$)"), ) COPYRIGHT_INDICATOR_RES: tuple[rex.Pattern[str], ...] = tuple( rex.compile(re_flag.re, re_flag.flags) for re_flag in COPYRIGHT_INDICATOR_REGEXES ) COPYRIGHT_INDICATOR_RE: rex.Pattern[str] = rex.compile( "|".join(f"{re_flag.re}" for re_flag in COPYRIGHT_INDICATOR_REGEXES), rex.IGNORECASE | rex.MULTILINE, ) COPYRIGHT_PRE_INDICATOR_RE: rex.Pattern[str] = rex.compile( r"Copyright|copr\.|©|\xa9|©|&\#169;|\(\s?c\s?\)", rex.IGNORECASE | rex.MULTILINE, ) COPYRIGHT_POST_IGNORE_REGEXES: tuple[ReFlags, ...] = ( # Discussing copyright information ReFlags(r"\binformation\b"), # Discussing the notice ReFlags(r"\bnotices?\b"), ReFlags(r"\bstatements?\b"), ReFlags(r"\bclaims?\b"), ReFlags(r"\bstrings?\b"), # GPL template ReFlags(r"\b\?\b"), # template ReFlags(r"\bYEAR\s+YOUR\s+NAME\b"), ReFlags(r"\bYour\s+Name\s+\"), ReFlags(r"{{copyright}}"), # Part of a sentence # |and|or used in: and/or its subsidiary(-ies). # |in used in .in tld ReFlags(r"\b\ in\ \b"), # |to used in "1990 to 1995" ReFlags(r"^\s*is\b"), ReFlags(r"\bif\b"), ReFlags(r"\bholders?\b"), ReFlags(r"\bowners?\b"), ReFlags(r"\bownership\b"), # template ReFlags(r"\bIDC_STATIC\b"), # (c) conditional ReFlags(r"(?:^|\s)&&(?:\s|$)", 0), ReFlags(r"(?:^|\s)\|\|(?:\s|$)", 0), # ugly one letter expressions ReFlags(r"(?:^|\s)\(\s?[abd-f]\s?\)(?:\s|$)"), # GPL text ReFlags(r"^\s*laws?[.,:]"), # if (c) { ReFlags(r"^\s*\{\s*$", 0), # Copyright: License ReFlags(r"^\s*L?GPL$\s*$"), # template ReFlags(r"^\s*@\w+\{\w+\}\s*$", 0), # copyright of Qt has been transferred ReFlags(r"^\s*of\ [^\s]+\ has\b"), # copyright applied to ReFlags(r"^\s*applied\ to\b"), # QPL false positive ReFlags(r"^\s*Everyone\ is\ permitted\ to\ copy\b"), ) COPYRIGHT_POST_IGNORE_RES: tuple[rex.Pattern[str], ...] = tuple( rex.compile(re_flag.re, re_flag.flags) for re_flag in COPYRIGHT_POST_IGNORE_REGEXES ) CRUFT_SUBS: tuple[ReSub, ...] = ( # Strip Markdown link syntax before name/year parsing ReSub(reo.compile(r"\[([^\]]+)\]\[[^\]]*\]"), r"\1"), # [text][ref] -> text ReSub(reo.compile(r"\[([^\]]+)\]\([^)]*\)"), r"\1"), # [text](url) -> text # "All rights reserved" ends the holder list: drop it and anything after it # on the line (e.g. "... All Rights Reserved. NO WARRANTY." -> holder only). ReSub( reo.compile( r"""(?:(?:some|all)? rights reserved""" r"""|(?:some|all) rights)\b.*""", reo.IGNORECASE, ), r"", ), ReSub(reo.compile(r"It can be distributed", reo.IGNORECASE), r""), ReSub(reo.compile(r"and contributors", reo.IGNORECASE), r""), # Drop the fsf address from the copyright ReSub(reo.compile(r"Franklin St(?:reet)?, Fifth Floor,?", reo.IGNORECASE), r""), ReSub(reo.compile(r"et al", reo.IGNORECASE), r""), ReSub(reo.compile(r"\band$", reo.IGNORECASE), r""), ReSub(reo.compile(r"\\$"), r""), ReSub(reo.compile(r"""[\s,.;\*#'"]*$"""), r""), ReSub(reo.compile(r"\(\sc\s\)", reo.IGNORECASE), r""), ReSub(reo.compile(r""), r""), # Generic tag,
, etc ReSub(reo.compile(r"^(?:of|and)\s", reo.IGNORECASE), r""), # Unicode replacement char, present in badly converted files ReSub(reo.compile(r"�"), r""), ReSub(reo.compile(r"\s{2,}"), r" "), ReSub(reo.compile(r"^\s+"), r""), # Strip leading assignment operator (e.g. "copyright = u'...'") ReSub(reo.compile(r"^=\s*"), r""), ReSub(reo.compile(r"\s+$"), r""), ReSub(reo.compile(r"\\@"), r"@"), ReSub(reo.compile(r"–"), r"-"), ) EMAIL_SUBS: tuple[ReSub, ...] = ( ReSub(reo.compile(r""), r""), ReSub(reo.compile(r"%20"), r" "), # (HT|X)ML ReSub(reo.compile(r"(^\s?\s?$)"), r" "), ReSub(reo.compile(r"<?"), r"<"), ReSub(reo.compile(r">?"), r">"), ReSub(reo.compile(r"@"), r"@"), ReSub(reo.compile(r"“?"), r'"'), ReSub(reo.compile(r"\(c\)$", reo.IGNORECASE), r""), ReSub(reo.compile(r"\\[nt]$", reo.IGNORECASE), r""), # Expensive fix for ") at the end of the string ReSub(reo.compile(r"((?P\()?(?(paren).*?|))(?(paren)|\)+)?$"), r"\1"), ReSub(reo.compile(r"\s+\(?(where|at|@)\)?\s+", reo.IGNORECASE), r"@"), ReSub(reo.compile(r"\(at\)", reo.IGNORECASE), r"@"), ReSub(reo.compile(r"\s+\(?do?[tm]\)?\s+", reo.IGNORECASE), r"."), # Ugly fix for >mail@example.com< ReSub(reo.compile(r"(?:^|(?<=\s))\s*\>\s*(?=\w(?:\w|[.-])*@)"), r"<"), ReSub(reo.compile(r"\<\s*$"), r">"), ReSub( reo.compile( r"(?:^|(?<=\s))\s*((?!\<)\w(?:\w|[.-])*@" r"?:\w(?:\w|-)+(?:\.\w(?:\w|-)+)+(?))\s*(?:(?=\s)|$)", ), r"<\1>", ), ReSub(reo.compile(r"\s\s+"), r" "), ReSub(reo.compile(r"^\s"), r""), ReSub(reo.compile(r"\s$"), r""), ) NAME_CRUFT_SUBS: tuple[ReSub, ...] = ( ReSub(reo.compile(r"", reo.IGNORECASE), r""), ReSub(reo.compile(r"^>", reo.IGNORECASE), r""), ReSub(reo.compile(r"<$", reo.IGNORECASE), r""), ReSub(reo.compile(r"\\[nt]$", reo.IGNORECASE), r""), ReSub(reo.compile(r"^\(\s*c\s*\)\s*", reo.IGNORECASE), r""), ReSub(reo.compile(r"^and$", reo.IGNORECASE), r""), ReSub(reo.compile(r"^and\s+(?:others|contributors)?\s*", reo.IGNORECASE), r""), ReSub(reo.compile(r"\bbut is\b.*$", reo.IGNORECASE), r""), ) ARTISTIC_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r'The "?Clarified "?Artistic License"? ' r"Preamble ", rex.IGNORECASE, ), "ClArtistic", ), ReSub( rex.compile( r"You may charge a reasonable copying fee for any " r"distribution of this Package. You may charge " r"any fee you choose for support of this Package. " r"You may not charge a fee for this Package itself. " r"However, you may distribute this Package in aggregate " r"with other \(possibly commercial\) programs as " r"part of a larger \(possibly commercial\) software " r"distribution provided that you do not advertise " r"this Package as a product of your own. You may embed " r"this Package[\' ]?s interpreter within an executable " r"of yours \(by linking\);? this shall be construed as " r"a mere form of aggregation, provided that the complete " r"Standard Version of the interpreter is so embedded." r".+" r"Aggregation of this Package with a commercial " r"distribution is always permitted provided that " r"the use of this Package is embedded; that is, " r"when no overt attempt is made to make this Package[\' ]?s " r"interfaces visible to the end user of the commercial " r"distribution. Such use shall not be construed as a " r"distribution of this Package.", rex.IGNORECASE, ), "Artistic-1-Perl", ), ReSub( rex.compile( r"Aggregation of this Package with a commercial " r"distribution is always permitted provided that " r"the use of this Package is embedded; that is, " r"when no overt attempt is made to make this Package[\' ]?s " r"interfaces visible to the end user of the commercial " r"distribution. Such use shall not be construed as a " r"distribution of this Package.", rex.IGNORECASE, ), "Artistic-1-cl8", ), ReSub( rex.compile( r'The "?Artistic License"? Preamble.*?' r"You may reuse parts of this Package in your own programs, " r"provided that you explicitly state where you got them from", rex.IGNORECASE | rex.DOTALL, ), "Artistic-dist", ), ReSub( rex.compile( r'The "?Artistic License"? ' r"Preamble " r"The intent of this document is to state the conditions under " r"which a Package may be copied, such that the Copyright Holder " r"maintains some semblance of artistic control over the " r"development of the package, while giving the users of the " r"package the right to use and distribute the Package in a " r"more[- ]or[- ]less customary fashion, plus the right to make " r"reasonable modifications.", rex.IGNORECASE, ), "Artistic-1", ), ReSub( rex.compile( r"The Net Boolean Public License " r"Version ([\d.]+)", rex.IGNORECASE, ), r"NBPL-\1", ), ReSub( rex.compile( r"The Open Group Test Suite License " r"Preamble", rex.IGNORECASE, ), r"OGTSL", ), ReSub( rex.compile( r"The OpenLDAP Public License " r"Version ([\d.]+)", rex.IGNORECASE, ), r"OLDAP-\1", ), ) BSD_SUBS: tuple[ReSub, ...] = ( # Specific BSD variants (checked before the generic clause-count rules # below); each anchors on a clause unique to that variant. # The three Sun/Oracle variants share the "ARE HEREBY EXCLUDED" disclaimer; # the nuclear clause (and its "licensed" wording) or its absence tells them # apart. Anchoring on that disclaimer keeps them off the otherwise-identical # -2014 nuclear form, which carries the standard BSD disclaimer instead. ReSub( rex.compile( r"ARE HEREBY EXCLUDED.*not designed, licensed or intended for use " r"in the design, construction, operation or maintenance of any " r"nuclear", rex.IGNORECASE, ), "BSD-3-clause-no-nuclear-license", ), ReSub( rex.compile( r"ARE HEREBY EXCLUDED.*not designed or intended for use in the " r"design, construction, operation or maintenance of any nuclear", rex.IGNORECASE, ), "BSD-3-clause-no-nuclear-warranty", ), ReSub( rex.compile( r"DAMAGES OR LIABILITIES SUFFERED BY LICENSEE AS A RESULT OF OR " r"RELATING TO USE, MODIFICATION OR DISTRIBUTION", rex.IGNORECASE, ), "BSD-3-clause-Sun", ), ReSub( rex.compile( r"this paragraph are duplicated in all such forms and that any " r"documentation, advertising materials", rex.IGNORECASE, ), "BSD-4.3TAHOE", ), ReSub( rex.compile(r"BSD-4-Clause-Shortened", rex.IGNORECASE), "BSD-4-clause-shortened", ), ReSub( rex.compile( r"BSD-4-Clause \(University of California-Specific\)", rex.IGNORECASE ), "BSD-4-clause-UC", ), ReSub(rex.compile(r"The Clear BSD License", rex.IGNORECASE), "BSD-3-clause-clear"), ReSub( rex.compile( r"Flex carries the copyright used for BSD software", rex.IGNORECASE ), "BSD-3-clause-flex", ), ReSub( rex.compile( r"subject to receipt of any required approvals from the " r"U\.S\. Dept\. of Energy", rex.IGNORECASE, ), "BSD-3-clause-LBNL", ), ReSub( rex.compile( r"not designed, licensed or intended for use in the design, " r"construction, operation or maintenance of any military", rex.IGNORECASE, ), "BSD-3-clause-no-military", ), ReSub( rex.compile( r"reproduce at minimum a disclaimer substantially similar to the " r"NO WARRANTY disclaimer", rex.IGNORECASE, ), "BSD-3-clause-acpica", ), ReSub( rex.compile( r"the entire permission notice in its entirety, including the " r"disclaimer of warranties", rex.IGNORECASE, ), "BSD-3-clause-Tso", ), ReSub( rex.compile( r"the following disclaimer listed in this license in the documentation", rex.IGNORECASE, ), "BSD-3-clause-Open-MPI", ), ReSub( rex.compile( r"Neither the name of the HP nor the names of its contributors", rex.IGNORECASE, ), "BSD-3-clause-HP", ), ReSub( rex.compile( r"All modifications to the source code must be clearly marked as such", rex.IGNORECASE, ), "BSD-Mark-Modifications", ), ReSub( rex.compile( r"must appear in all copies of the software, derivative works or " r"modified versions", rex.IGNORECASE, ), "BSD-Inferno-Nettverk", ), ReSub( rex.compile( r"FREE FOR COMMERCIAL AND NON-COMMERCIAL USE as long as the " r"following conditions are adhered to", rex.IGNORECASE, ), "BSD-Systemics", ), ReSub( rex.compile( r"each copyright holder and contributor hereby grants to those " r"receiving rights under this license", rex.IGNORECASE, ), "BSD-2-clause-patent", ), ReSub( rex.compile( r"not subject to any export provision of the United States " r"Department of Commerce, and may be exported to any country or planet", rex.IGNORECASE, ), "BSD-2-clause-Darwin", ), ReSub( rex.compile( r"the following disclaimer, without modification, immediately at " r"the beginning of the file", rex.IGNORECASE, ), "BSD-source-beginning-file", ), ReSub( rex.compile( r"Redistributions of source code must retain the above copyright " r"notice, this list of conditions and the following disclaimer as " r"the first lines of this file", rex.IGNORECASE, ), "BSD-2-clause-first-lines", ), ReSub( rex.compile( r"this list of conditions and the following disclaimer\.\s*" r"THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS", rex.IGNORECASE, ), "BSD-1-clause", ), ReSub( rex.compile( r"4. If you include any Windows specific code \(or a derivative " r"thereof\) fromg? the apps directory \(application code\) you " r"must include an acknowledgement:? " r'"?This product includes software written by Tim Hudson ' r'\(tjh@cryptsoft.com\)"?', rex.IGNORECASE, ), "SSLeay", ), ReSub( rex.compile( r"Redistributions in any form must be accompanied by " r"information on how to obtain complete source code for the " r"software that uses (?:[^ ]+ ){,4}and any accompanying " r"software that uses the software that uses (?:[^ ]+ ){,4}" r"The source code must either be included in the " r"distribution or be available for no more than the cost of " r"distribution plus a nominal fee, and must be freely " r"redistributable under reasonable conditions\. For an " r"executable file, complete source code means the source code " r"for all modules it contains\. It does not include source code " r"for modules or files that typically accompany the major " r"components of the operating system on which the executable " r"file runs\. ", rex.IGNORECASE, ), "BSD-like-TMate", ), ReSub( rex.compile( r"with the name\(s\) of the authors as specified in the " r"copyright notice\(s\) substituted where indicated", rex.IGNORECASE, ), "BSD-like-radvd", ), ReSub( rex.compile( r"All advertising materials mentioning features or use of " r"this software must display the following " r"acknowledge?ment.*This product includes software " r"developed by", rex.IGNORECASE, ), "BSD-4-clause", ), ReSub( rex.compile( r"You acknowledge that this software is not designed" r"(?:, licensed)? " r"or intended for use in the design, construction, operation or " r"maintenance of any nuclear facility.", rex.IGNORECASE, ), "BSD-3-clause-no-nuclear", ), ReSub( rex.compile( r"Redistributions of source code must retain the above " r"copyright notice, this list of conditions and the following " r"disclaimer.? " r"(?:\* )?" r"(?:The name(?:\(?s\)?)? .*? may not|Neither the " r"(?:names? .*?|authors?) ?n?or the names of(?: " r"(?:its|their|other|any))? contributors may) be used to " r"endorse or promote products derived from this software", rex.IGNORECASE, ), "BSD-source-code", ), ReSub( rex.compile( r"Redistributions of any form whatsoever must retain the " r"following acknowledgment:? \'?This product includes software " r"developed by ", rex.IGNORECASE, ), "BSD-3-clause-attribution", ), # bzip2 carries its version in the notice; name the two shipped variants # before the generic bzip2 rule below. ReSub( rex.compile(r"Version 1\.0\.5 of", rex.IGNORECASE), "BSD-like-bzip2-1.0.5", ), ReSub( rex.compile(r"version 1\.0\.6 of", rex.IGNORECASE), "BSD-like-bzip2-1.0.6", ), ReSub( rex.compile( r"The origin of this software must not be misrepresented;? " r"you must not claim that you wrote the original software. " r"If you use this software in a product, an acknowledgment " r"in the product documentation would be appreciated but is " r"not required." r".+" r"Altered source versions must be plainly marked as such, " r"and must not be misrepresented as being the original " r"software." r".+" r"The name of the author may not be used to endorse or " r"promote products derived from this software without " r"specific prior written permission.", rex.IGNORECASE, ), "BSD-like-bzip2", ), ReSub(rex.compile(r"Intel Open Source License ", rex.IGNORECASE), "Intel"), ReSub( rex.compile( r"All redistributions must comply with the conditions imposed by " r"the University of California on certain embedded code, which " r"copyright Notice and conditions for redistribution are as " r"follows", rex.IGNORECASE, ), "Sendmail", ), ReSub( rex.compile( r"Redistributions in any form must be accompanied by " r"information on how to obtain complete source code for " r"the DB software and any accompanying software that uses the " r"DB software\. The source code must either be included in the " r"distribution or be available for no more than the cost of " r"distribution plus a nominal fee, and must be freely " r"redistributable under reasonable conditions\. For an " r"executable file, complete source code means the source code " r"for all modules it contains\. It does not include source code " r"for modules or files that typically accompany the major " r"components of the operating system on which the executable " r"file runs\.", rex.IGNORECASE, ), "Sleepycat", ), ReSub( rex.compile( r"Any altered version must clearly and properly represent the " r"origin of this software", rex.IGNORECASE, ), "BSD-like-ngrep", ), ReSub( rex.compile( r"Part 1: CMU/?UCD copyright notice", rex.IGNORECASE, ), "Net-SNMP", ), ReSub( rex.compile( r"Use, Modification and Redistribution \(including distribution " r"of any modified or derived work\) in source and/?or binary " r"forms is permitted", rex.IGNORECASE, ), "BSD-like-JPNIC", ), ReSub( rex.compile( r"(?:The name(?:\(?s\)?)? .*? may not|Neither the " r"(?:names? .*?|authors?) n?or the names of(?: " r"(?:its|their|other|any))? contributors may) be used to " r"endorse or promote products derived from this software", rex.IGNORECASE, ), "BSD-3-clause", ), ReSub( rex.compile( r"Any additions, deletions, or changes to the original files " r"must be clearly indicated in accompanying documentation\. " r"including the reasons for the changes, and the names of " r"those who made the modifications\.", rex.IGNORECASE, ), "BSD-like-Mup", ), ReSub( rex.compile( r"Redistributions in any form must be accompanied by information " r"on how to obtain complete source code for this software and " r"any accompanying software that uses this software\. The source " r"code must either be included in the distribution or be " r"available in a timely fashion for no more than the cost of " r"distribution plus a nominal fee, and must be freely " r"redistributable under reasonable and no more restrictive " r"conditions\. For an executable file, complete source code " r"means the source code for all modules it contains\. It does " r"not include source code for modules or files that typically " r"accompany the major components of the operating system on " r"which the executable file runs\. ", rex.IGNORECASE, ), "BSD-like-TOSL", ), ReSub( rex.compile( r"The views and conclusions contained in the software and " r"documentation are those of the authors and should not " r"be interpreted as representing official policies, " r"either expressed or implied, of the FreeBSD Project.", rex.IGNORECASE, ), "BSD-2-clause-FreeBSD", ), ReSub( rex.compile( r"views and conclusions contained in the software and " r"documentation are those of the authors and should not be " r"interpreted as representing official policies", rex.IGNORECASE, ), "BSD-2-clause-views", ), ReSub( rex.compile( r"This code is derived from software contributed to The " r"NetBSD Foundation by", rex.IGNORECASE, ), "BSD-2-clause-NetBSD", ), ReSub( rex.compile( r"Nothing in this license shall be deemed to grant " r"any rights to trademarks, copyrights, patents, trade " r"secrets or any other intellectual property of A.M.P.A.S. " r"or any contributors, except as expressly stated herein, " r"and neither the name of A.M.P.A.S. nor of any other " r"contributors to this software, may be used to endorse " r"or promote products derived from this software without " r"specific prior written permission of A.M.P.A.S. or " r"contributor, as appropriate.", rex.IGNORECASE, ), "BSD-2-clause-AMPAS", ), ReSub( rex.compile( r"This code is derived from software contributed to The " r"NetBSD Foundation by", rex.IGNORECASE, ), "BSD-2-clause-NetBSD", ), ReSub( rex.compile( r"Redistributions of source code must retain the(?: above)? " r"copyright notice", rex.IGNORECASE, ), "BSD-2-clause", ), ReSub( rex.compile( r"BSD PROTECTION LICENSE " r"TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION, AND " r"MODIFICATION", rex.IGNORECASE, ), "BSD-protection", ), ReSub( rex.compile( r"Permission to use, copy, modify, and/?or distribute this " r"software for any purpose with or without fee is hereby " r"granted. " r'THE SOFTWARE IS PROVIDED "?AS IS"? AND THE AUTHOR DISCLAIMS ' r"ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL " r"IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO " r"EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, " r"INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER " r"RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN " r"ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, " r"ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE " r"OF THIS SOFTWARE.", rex.IGNORECASE, ), "0BSD", ), ReSub( rex.compile( r"Redistributions of the Code in binary form must be " r"accompanied by this GPG-signed text in any documentation " r"and, each time the resulting executable program or a " r"program dependent thereon is launched, a prominent " r"display \(?e\.g\., splash screen or banner text\)? " r"of the Author[\' ]?s attribution information, which " r"includes:? " r'\(?a\)? Name \(?[" ]AUTHOR[" ]\)?, ' r'\(?b\)? Professional identification \(?[" ]PROFESSIONAL ' r'IDENTIFICATION[" ]\)?, and ' r'\(?c\)? URL \(?[" ]URL[" ]\)?.', rex.IGNORECASE, ), "AAL", ), ReSub( rex.compile( r"The Condor Team may publish revised and/or new versions " r"of this Condor Public License " r'\((?:``|[" ])?this License(?:\'\'|[" ])?\) ' r"from time to time\. Each " r"version will be given a distinguishing version number\. " r"Once Software has been published under a particular " r"version of this License, you may always continue to use " r"it under the terms of that version\. You may also " r"choose to use such Software under the terms of " r"any subsequent version of this License published " r"by the Condor Team\. No one other than the Condor " r"Team has the right to modify the terms of this License\.", rex.IGNORECASE, ), "Condor-1.1", ), ReSub( rex.compile( r"This product includes software developed by the EU DataGrid", rex.IGNORECASE, ), "EUDatagrid", ), ReSub( rex.compile( r"Access and use of this software shall impose the following " r"obligations and understandings on the user\. The user is " r"granted the right, without any fee or cost, to use, copy, " r"modify, alter, enhance and distribute this software, and " r"any derivative works thereof, and its supporting " r"documentation for any purpose whatsoever, provided that " r"this entire notice appears in all copies of the software, " r"derivative works and supporting documentation\. Further, " r"(?:[^\s]+ ){,8}requests that the user credit " r"(?:[^\s]+ ){,8}in any publications that result from the " r"use of this software or in any product that includes " r"this software, although this is not an obligation\. The " r"names " r"(?:[^\s]+ ){,8}however, may not be used in any advertising " r"or publicity to endorse or promote any products or " r"commercial entity unless specific written permission is " r"obtained from " r"(?:[^\s]+ ){,8}The user also understands that " r"(?:[^\s]+ ){,8}is not obligated to provide the user with any " r"support, consulting, training or assistance of any kind with " r"regard to the use, operation and performance of this software " r"nor to provide the user with any updates, revisions, new " r'versions or "?bug fixes', rex.IGNORECASE, ), "NetCDF", ), ReSub( rex.compile( r"You may make and give away verbatim copies of the source " r"form of the software without restriction, provided that you " r"duplicate all of the original copyright notices and " r"associated disclaimers\. " r".{,4}You may modify your copy of the software in any way, " r"provided that you do at least ONE of the following:? ", rex.IGNORECASE, ), "Ruby", ), ReSub( rex.compile( r"The copyright holder has authorized the removal of this clause", rex.IGNORECASE, ), "Inner-Net-2.0", ), ReSub( rex.compile( r"Redistribution and use in source and binary forms -? ?with or " r"without modification -? ?are permitted for any purpose, " r"provided that redistributions in source form retain this " r"entire copyright notice and indicate the origin and nature of " r"any modifications\. " r"I\'d appreciate being given credit for this package in the " r"documentation of software which uses it, but that is not a " r"requirement\. ", rex.IGNORECASE, ), "Spencer-99", ), ) # Title prefix shared by the CC_BY_SUBS rules. The "Creative Commons"/"CC" lead # is optional (some ports open with a bare "Attribution-…" title), and the mod # tokens tolerate the hyphenated/spaced spellings used by the jurisdiction ports # ("Non-Commercial", "Share-Alike", "Attribution - Share-Alike 2.0 England"). # The "Attribution" head also accepts the localized spellings used by the # non-English ports (German Namensnennung, Dutch Naamsvermelding, French # Paternité, Japanese katakana アトリビューション); clean_comments() turns the # em-dash separators those titles use into spaces, so [ -] still covers them. _CC_BY = r"(?:Attribution|Namensnennung|Naamsvermelding|Paternité|アトリビューション)" _CC_TITLE = ( r"(?:https?[: ]/?/?creativecommons.org/licenses/by" rf"|(?:(?:Creative Commons|CC) (?:Legal Code )?)?{_CC_BY})[ -]+" ) # Modifier tokens carry their localized spellings too. German uses two NC # spellings across versions ("Nicht-kommerziell" v2.0 / "Keine kommerzielle # Nutzung" v3.0); list the longer phrases before the bare "nc"/"sa"/"nd". _CC_NC = ( r"(?:Non-?Commercial|Nicht-kommerziell|Keine kommerzielle Nutzung" r"|Pas d'Utilisation Commerciale|nc)" ) _CC_SA = ( r"(?:Share-?Alike|Weitergabe unter gleichen Bedingungen" r"|Partage Des Conditions Initiales A l'Identique|シェアアライク|sa)" ) _CC_ND = r"(?:No-?Deriv(?:atives|s)?|Keine Bearbeitungen?|nd)" CC_BY_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile( _CC_TITLE + rf"(?:(?:{_CC_SA}|{_CC_ND})[ -]+){{,2}}{_CC_NC}", rex.IGNORECASE, ), r"-NC", ), ReSub( rex.compile( _CC_TITLE + rf"(?:(?:{_CC_NC}|{_CC_ND})[ -]+){{,2}}{_CC_SA}", rex.IGNORECASE, ), r"-SA", ), ReSub( rex.compile( _CC_TITLE + rf"(?:(?:{_CC_NC}|{_CC_SA})[ -]+){{,2}}{_CC_ND}", rex.IGNORECASE, ), r"-ND", ), ReSub( rex.compile( _CC_TITLE + rf"(?:(?:{_CC_NC}|{_CC_SA}|{_CC_ND})[ -]*){{,2}}/?" r"(\d(?:\.\d[\d.]*)?)[^\d.]", rex.IGNORECASE, ), r"-\1", ), # Jurisdiction port suffix, appended after the version (e.g. CC-BY-3-US). # A port has exactly one jurisdiction, so this is a first-match-wins group. # Each marker is a phrase distinctive to that jurisdiction's legal code; the # NL/FR ports carry no country name in the text, so the localized language # word (Dutch "Naamsvermelding", French "Paternité") stands in for the single # jurisdiction shipped in that language. ReSubGroup( ( ReSub(rex.compile(r"United States", rex.IGNORECASE), r"-US"), ReSub(rex.compile(r"England and Wales", rex.IGNORECASE), r"-UK"), ReSub(rex.compile(r"Australia", rex.IGNORECASE), r"-AU"), ReSub(rex.compile(r"intergovernmental", rex.IGNORECASE), r"-IGO"), ReSub(rex.compile(r"Deutschland", rex.IGNORECASE), r"-DE"), ReSub(rex.compile(r"Naamsvermelding", rex.IGNORECASE), r"-NL"), ReSub(rex.compile(r"Paternité", rex.IGNORECASE), r"-FR"), ReSub(rex.compile(r"ジャパン", rex.IGNORECASE), r"-JP"), ), ), ) # Documentation / open-government / open-data licences (run by parse_doc only # after the DocLicense gate in LICENSES_RES fires). Each rule pairs the title # with a substantive clause from the licence body, so a file that only mentions # the title (e.g. a reference or a `License:` field) is not matched. First match # wins. The UK OGL / OPL / NCGL family shares one grant clause; the title tells # them apart. _OGL_GRANT = r"grants you a worldwide, royalty-free, perpetual, non-exclusive licence" DOC_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( rf"Open Government Licence v1\.0.{{0,400}}{_OGL_GRANT}", rex.IGNORECASE, ), "OGL-UK-1", ), ReSub( rex.compile( rf"Open Government Licence v2\.0.{{0,400}}{_OGL_GRANT}", rex.IGNORECASE, ), "OGL-UK-2", ), ReSub( rex.compile( rf"Open Government Licence v3\.0.{{0,400}}{_OGL_GRANT}", rex.IGNORECASE, ), "OGL-UK-3", ), ReSub( rex.compile( rf"Open Government Licence - Canada.{{0,400}}{_OGL_GRANT}", rex.IGNORECASE ), "OGL-Canada-2", ), ReSub( rex.compile( rf"Open Parliament Licence v3\.0.{{0,400}}{_OGL_GRANT}", rex.IGNORECASE, ), "OPL-UK-3", ), ReSub( rex.compile( r"Non-Commercial Government Licence for public sector information" rf".{{0,400}}{_OGL_GRANT}", rex.IGNORECASE, ), "NCGL-UK-2", ), ReSub( rex.compile( r"Norwegian Licence for Open Government Data \(NLOD\) 2\.0.{0,200}" r"This licence grants you the right to copy, use and distribute " r"information", rex.IGNORECASE, ), "NLOD-2", ), ReSub( rex.compile( r"LICENCE OUVERTE / OPEN LICENCE Version 2\.0.{0,200}concède au " r"Réutilisateur un droit non exclusif et gratuit de libre " r"Réutilisation", rex.IGNORECASE, ), "etalab-2", ), ReSub( rex.compile( r"Common Documentation License Version 1\.0.{0,200}Permission is " r"granted to copy and distribute verbatim copies of this License", rex.IGNORECASE, ), "CDL-1", ), ReSub( rex.compile( r"This is the Community Data License Agreement Permissive, " r"Version 1\.0 \( Agreement \)\. Data is provided to You under " r"this Agreement", rex.IGNORECASE, ), "CDLA-Permissive-1", ), ReSub( rex.compile( r"This is the Community Data License Agreement - Permissive, " r"Version 2\.0 \(the agreement \)\. Data Provider\(s\) and Data " r"Recipient\(s\) agree as follows", rex.IGNORECASE, ), "CDLA-Permissive-2", ), ReSub( rex.compile( r"This is the Community Data License Agreement Sharing, " r"Version 1\.0 \( Agreement \)\. Data is provided to You under " r"this Agreement", rex.IGNORECASE, ), "CDLA-Sharing-1", ), ReSub( rex.compile( r"The Open Data Commons Attribution License is a license " r"agreement intended to allow users to freely share, modify, and " r"use this Database", rex.IGNORECASE, ), "ODC-By-1", ), ReSub( rex.compile( r"Jede Nutzung ist unter den Bedingungen dieser Datenlizenz " r"Deutschland Namensnennung Version 2\.0 zul", rex.IGNORECASE, ), "DL-DE-BY-2", ), ReSub( rex.compile( r"Datenlizenz Deutschland Zero Version 2\.0 Jede Nutzung ist ohne " r"Einschränkungen oder Bedingungen zul", rex.IGNORECASE, ), "DL-DE-Zero-2", ), ReSub( rex.compile( r"Open Publication License v1\.0.{0,80}REQUIREMENTS ON BOTH " r"UNMODIFIED AND MODIFIED VERSIONS The Open Publication works may " r"be reproduced and distributed", rex.IGNORECASE, ), "OPUBL-1", ), ) # Font licences (run by parse_font only after the FontLicense gate in # LICENSES_RES fires). Each rule anchors on a substantive grant/permission # clause from the font licence body, not merely the foundry or font name, so a # file that just mentions a font is not matched. First match wins. FONT_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"ARPHIC PUBLIC LICENSE specifically permits and encourages you " r"to use this software", rex.IGNORECASE, ), "Arphic-1999", ), ReSub( rex.compile( r"Permission to use, copy, modify and distribute this font is " r"hereby granted, provided that both the copyright notice and this " r"permission notice appear in all copies of the font", rex.IGNORECASE, ), "Baekmuk", ), ReSub( rex.compile( r"granted permission under all Bitstream propriety rights to use, " r"copy, modify, sublicense, sell, and redistribute", rex.IGNORECASE, ), "Bitstream-Charter", ), ReSub( rex.compile( r"obtaining a copy of the fonts accompanying this license " r"\( Fonts \) and associated documentation files " r"\(the Font Software \)", rex.IGNORECASE, ), "Bitstream-Vera", ), ReSub( rex.compile( r"Any use of the code is permitted as long as this copyright " r"notice is preserved in the code", rex.IGNORECASE, ), "Kastrup", ), ReSub( rex.compile( r"are hereby granted a nonexclusive, royalty-free copyright and " r"design patent license to use this code", rex.IGNORECASE, ), "Lucida-Bitmap-Fonts", ), ReSub( rex.compile( r"ParaType Ltd grants you the right to use, copy, modify the fonts " r"and distribute modified and unmodified copies", rex.IGNORECASE, ), "ParaType-Free-Font-1.3", ), ReSub( rex.compile( r"Unlimited permission is granted to use, copy, and distribute it, " r"with or without modification, either commercially and " r"noncommercially", rex.IGNORECASE, ), "mplus", ), ReSub( rex.compile( r"UBUNTU FONT LICENCE Version 1\.0.{0,200}This licence allows the " r"licensed fonts to be used, studied, modified and redistributed " r"freely", rex.IGNORECASE, ), "Ubuntu-font-1", ), ) # Open-hardware licences (run by parse_hw only after the HardwareLicence gate in # LICENSES_RES fires). Each rule pairs the versioned title with a substantive # preamble clause; the CERN v2 Permissive/Strongly/Weakly variants share one # preamble and are told apart by the variant word in the title. First match wins. _CERN_V2 = r"freedom to use, study, modify, share and distribute hardware designs" HW_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"CERN Open Hardware Licence v1\.1.{0,400}wishes to disseminate its " r"hardware designs", rex.IGNORECASE, ), "CERN-OHL-1.1", ), ReSub( rex.compile( r"CERN Open Hardware Licence v1\.2.{0,400}wishes to provide a tool " r"to foster collaboration", rex.IGNORECASE, ), "CERN-OHL-1.2", ), ReSub( rex.compile( rf"CERN Open Hardware Licence Version 2 - Permissive.{{0,400}}{_CERN_V2}", rex.IGNORECASE, ), "CERN-OHL-P-2", ), ReSub( rex.compile( r"CERN Open Hardware Licence Version 2 - Strongly Reciprocal" rf".{{0,400}}{_CERN_V2}", rex.IGNORECASE, ), "CERN-OHL-S-2", ), ReSub( rex.compile( r"CERN Open Hardware Licence Version 2 - Weakly Reciprocal" rf".{{0,400}}{_CERN_V2}", rex.IGNORECASE, ), "CERN-OHL-W-2", ), ReSub( # The title page ("based closely on the Apache License") or the standard # licence header ("you may not use this file except in compliance"); both # are version-specific, so the header is detected as well as the body. rex.compile( r"SOLDERPAD HARDWARE LICENSE version 0\.5 .{0,200}based closely on " r"the Apache License Version 2\.0" r"|Solderpad Hardware License, Version 0\.5(?!\d).{0,160}you may not " r"use this file except in compliance", rex.IGNORECASE, ), "SHL-0.5", ), ReSub( rex.compile( r"SOLDERPAD HARDWARE LICENSE version 0\.51.{0,200}based closely on " r"the Apache License Version 2\.0" r"|Solderpad Hardware License, Version 0\.51.{0,160}you may not " r"use this file except in compliance", rex.IGNORECASE, ), "SHL-0.51", ), ReSub( rex.compile( r"The TAPR Open Hardware License Version 1\.0.{0,400}Open Hardware " r"is a thing", rex.IGNORECASE, ), "TAPR-OHL-1", ), ) # US-Government / agency public-domain notices (run by parse_usgov_pd only after # the USGovPD gate in LICENSES_RES fires). Several share the "Section 105 / works # of employees are not subject to copyright" clause, so each rule pins # the substantive public-domain clause together with the agency that names the # id (e.g. NIST-PD vs NTIA-PD differ only by agency). First match wins. USGOV_PD_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"This software was developed by employees of the National " r"Institute of Standards and Technology \(NIST\), and others\. " r"This software has been contributed to the public domain", rex.IGNORECASE, ), "NIST-PD", ), ReSub( rex.compile( r"This software was developed by employees of the National " r"Institute of Standards and Technology \(NIST\), an agency of the " r"Federal Government and is being made available as a public " r"service", rex.IGNORECASE, ), "NIST-PD-fallback", ), ReSub( rex.compile( r"This software was developed at the National Institute of " r"Standards and Technology \(NIST\) by employees of the Federal " r"Government in the course of their official duties", rex.IGNORECASE, ), "NIST-PD-TNT", ), ReSub( rex.compile( r"NIST-developed software is provided by NIST as a public " r"service\. You may use, copy, and distribute copies of the " r"software in any medium, provided that you keep intact this " r"entire notice", rex.IGNORECASE, ), "NIST-Software", ), ReSub( rex.compile( r"developed by employees of the National Telecommunications and " r"Information Administration \(NTIA\), an agency of the Federal " r"Government and is provided to you as a public service", rex.IGNORECASE, ), "NTIA-PD", ), ReSub( rex.compile( r"PUBLIC DOMAIN NOTICE National Center for Biotechnology " r"Information", rex.IGNORECASE, ), "NCBI-PD", ), ReSub( rex.compile( r"This library \(libselinux\) is public domain software, i\.e\. " r"not copyrighted", rex.IGNORECASE, ), "libselinux-1", ), ) # Miscellaneous public-domain / "do as you wish" notices that share no common # origin (run by parse_pd_misc only after the PdMisc gate in LICENSES_RES fires). # Each rule anchors on the notice's distinctive dedication clause. First match # wins. The SAX v2 form starts "SAX2 is Free", which keeps it off the v1 SAX-PD # (already detected) that embeds the same "abandon ... SAX 2.0" sentence. PD_MISC_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile(r"In place of a legal notice, here is a blessing", rex.IGNORECASE), "blessing", ), ReSub( rex.compile( r"In case this attempt to disclaim copyright and place the " r"software in the public domain is deemed null and void", rex.IGNORECASE, ), "bcrypt-Solar-Designer", ), ReSub( rex.compile( r"SAX2 is Free I hereby abandon any property rights to SAX 2\.0", rex.IGNORECASE, ), "SAX-PD-2", ), ReSub( rex.compile( r"There is no license associated with the code and you may use it " r"for any purpose personal or commercial", rex.IGNORECASE, ), "SGP4", ), ReSub( rex.compile( r"My symlinks utility pre-dates the open source licensing fad by a " r"number of years", rex.IGNORECASE, ), "Symlinks", ), ReSub( rex.compile( r"DO WHAT THE FUCK YOU WANT TO BUT IT'S NOT MY FAULT PUBLIC LICENSE", rex.IGNORECASE, ), "WTFNMFPL", ), ReSub( rex.compile( r"dedicate any and all copyright interest in this code to the " r"public domain", rex.IGNORECASE, ), "Unlicense-libtelnet", ), ReSub( rex.compile( r"LZMA SDK is written and placed in the public domain by Igor " r"Pavlov\. Some code in LZMA is based on public domain code", rex.IGNORECASE, ), "LZMA-SDK-9.11-to-9.20", ), ReSub( rex.compile( r"Some code in LZMA SDK is based on public domain code from " r"another developers", rex.IGNORECASE, ), "LZMA-SDK-9.22", ), ) # Linux man-pages licences (run by parse_manpage only after the ManPage gate in # LICENSES_RES fires). The three "copyleft" variants share the same two-paragraph # FSF manual permission; the specific tails (the "level of care" sentence, the # "...herein. Formatted" sequence) come first, and the bare two-paragraph form is # the last fallback. Its negative lookahead skips the Latex2e family, which adds # a "translations of this manual" paragraph after the same two paragraphs. MANPAGE_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"distribute possibly modified copies of this page provided the " r"header is included verbatim", rex.IGNORECASE, ), "Linux-man-pages-1-para", ), ReSub( rex.compile( r"may not have taken the same level of care in the production of " r"this manual, which is licensed free of charge", rex.IGNORECASE, ), "Linux-man-pages-copyleft", ), ReSub( rex.compile( r"for damages resulting from the use of the information contained " r"herein\. Formatted or processed versions", rex.IGNORECASE, ), "Linux-man-pages-copyleft-var", ), ReSub( rex.compile( r"make and distribute verbatim copies of this manual provided the " r"copyright notice and this permission notice are preserved" r".{0,300}permission notice identical to this one\." r"(?!\s*Permission is granted to copy and distribute translations)", rex.IGNORECASE, ), "Linux-man-pages-copyleft-2-para", ), ) # CTAN / TeX licences (run by parse_ctan only after the CtanTex gate in # LICENSES_RES fires). These share the TeX/CTAN ecosystem but no single clause; # each rule anchors on its distinctive permission/rename clause. First match # wins. CTAN_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"Unlimited copying and redistribution of this package and/?or its " r"individual files are permitted as long as there are no " r"modifications", rex.IGNORECASE, ), "Knuth-CTAN", ), ReSub( rex.compile( r"This file may be freely copied and distributed, provided that no " r"changes whatsoever are made\. All users are asked to help keep", rex.IGNORECASE, ), "MMIXware", ), ReSub( rex.compile( r"except that this permission notice may be stated in a " r"translation approved by the Foundation", rex.IGNORECASE, ), "Latex2e-translated-notice", ), ReSub( rex.compile( r"This software may be freely used, transmitted, reproduced, or " r"modified provided that the copyright notice and this permission " r"is retained", rex.IGNORECASE, ), "magaz", ), ReSub( rex.compile( r"The style package is copyrighted but may be used and extended in " r"any way, as long as a pointer to the original author is " r"maintained", rex.IGNORECASE, ), "swrule", ), ReSub( rex.compile( r"This file may be distributed, modified, and used in other works " r"with just one restriction: modified versions must clearly " r"indicate the modification", rex.IGNORECASE, ), "threeparttable", ), ReSub( rex.compile( r"This software may be freely transmitted, reproduced, or modified " r"for any purpose provided that this copyright notice is left " r"intact", rex.IGNORECASE, ), "ulem", ), ReSub( rex.compile( r"THE TTYP0 LICENSE Permission is hereby granted, free of charge, " r"to any person obtaining a copy of this font software", rex.IGNORECASE, ), "TTYP0", ), ) # Standards-body licences - Unicode, plus the W3C-derived family (run by # parse_std only after the StandardsBody gate fires). The gate deliberately omits # the "W3C SOFTWARE NOTICE AND LICENSE" title, so the already-detected W3C / # W3C-19980720 are untouched. The two Unicode DFS years differ only by the ICU # directory (2016), so the ICU rule precedes the bare-title 2015 fallback. First # match wins. STD_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"UNICODE LICENSE V3 COPYRIGHT AND PERMISSION NOTICE", rex.IGNORECASE ), "Unicode-3", ), ReSub( rex.compile( r"UNICODE, INC\. LICENSE AGREEMENT - DATA FILES AND SOFTWARE" r".{0,400}source\.icu-project\.org", rex.IGNORECASE, ), "Unicode-DFS-2016", ), ReSub( rex.compile( r"UNICODE, INC\. LICENSE AGREEMENT - DATA FILES AND SOFTWARE", rex.IGNORECASE, ), "Unicode-DFS-2015", ), ReSub( rex.compile(r"OGC Software License, Version 1\.0", rex.IGNORECASE), "OGC-1", ), ReSub( rex.compile( r"WordNet Release 3\.0 This software and database is being " r"provided to you, the LICENSEE, by Princeton University", rex.IGNORECASE, ), "WordNet", ), ReSub( rex.compile( r"By obtaining and/?or copying this work, you \(the licensee\) " r"agree", rex.IGNORECASE, ), "W3C-20150513", ), ) # Source-available / business copyleft licences - each carries a distinctive # title-and-version line, paired here with a substantive body clause (release # date, canonical URL, or the licence's own summary sentence) so a bare # reference to the name does not match. parse_srcavail names the variant after # the gate fires. The two Parity versions share a title stem but differ in their # one-line summary; the two copyleft-next releases differ only by release date. # Single-".0"-suffixed ids let the family parser strip the trailing ".0" # (SSPL-1.0 -> SSPL-1, Elastic-2.0 -> Elastic-2); ids with multiple trailing # zero components carry the fully-stripped name as the repl (Parity-6.0.0 -> # Parity-6, PolyForm-Noncommercial-1.0.0 -> PolyForm-Noncommercial-1). SRCAVAIL_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"copyleft-next 0\.3\.0 \( this License \) " r"Release date: 2013-05-16", rex.IGNORECASE, ), "copyleft-next-0.3.0", ), ReSub( rex.compile( r"copyleft-next 0\.3\.1 \( this License \) " r"Release date: 2016-04-29", rex.IGNORECASE, ), "copyleft-next-0.3.1", ), ReSub( rex.compile( r"The Parity Public License 6\.0\.0.{0,200}This license lets you " r"use and share this software for free, as long as you contribute", rex.IGNORECASE, ), "Parity-6", ), ReSub( rex.compile( r"The Parity Public License 7\.0\.0.{0,200}This license allows you " r"to use and share this software for free, but you have to share " r"software that builds on it alike", rex.IGNORECASE, ), "Parity-7", ), ReSub( rex.compile( r"PolyForm Noncommercial License 1\.0\.0 " r"https://polyformproject\.org/licenses/noncommercial", rex.IGNORECASE, ), "PolyForm-Noncommercial-1", ), ReSub( rex.compile( r"PolyForm Small Business License 1\.0\.0 " r"https://polyformproject\.org/licenses/small-business", rex.IGNORECASE, ), "PolyForm-Small-Business-1", ), ReSub( rex.compile( r"Server Side Public License VERSION 1, OCTOBER 16, 2018", rex.IGNORECASE, ), "SSPL-1.0", ), # SSPL standardLicenseHeader: the application notice grants under the SSPL # by reference ("under the terms of the Server Side Public License, # version 1"). ReSub( rex.compile( r"under the terms of the Server Side Public License, version 1", rex.IGNORECASE, ), "SSPL-1.0", ), ReSub( rex.compile( r"Business Source License 1\.1 License text copyright", rex.IGNORECASE ), "BUSL-1.1", ), # BUSL standardLicenseHeader: the application notice puts use under the BUSL # by reference (verbatim typo "govered" from the upstream template). ReSub( rex.compile( r"Use of this software is govered by the Business Source License", rex.IGNORECASE, ), "BUSL-1.1", ), ReSub( rex.compile( r"Elastic License 2\.0 URL: " r"https://www\.elastic\.co/licensing/elastic-license", rex.IGNORECASE, ), "Elastic-2.0", ), ReSub( rex.compile( r"Sustainable Use License Version 1\.0 Acceptance By using the " r"software, you agree to all of the terms", rex.IGNORECASE, ), "SUL-1.0", ), ) # European Space Agency Public License - three copyleft strengths sharing one # template (run by parse_esa_pl only after the EsaPl gate fires). Each rule # pairs the variant's title-and-version line with the start of the shared # Definitions clause, so a bare reference to the family name declines. ESA_PL_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"European Space Agency Public License \(ESA-PL\) Permissive " r"\(Type 3\) v2\.4 1 Definitions", rex.IGNORECASE, ), "ESA-PL-permissive-2.4", ), ReSub( rex.compile( r"European Space Agency Public License \(ESA-PL\) Weak Copyleft " r"\(Type 2\) v2\.4 1 Definitions", rex.IGNORECASE, ), "ESA-PL-weak-copyleft-2.4", ), ReSub( rex.compile( r"European Space Agency Public License \(ESA-PL\) Strong Copyleft " r"\(Type 1\) v2\.4 1 Definitions", rex.IGNORECASE, ), "ESA-PL-strong-copyleft-2.4", ), ) # TU-Berlin (GSM 06.10) notice - v2.0 is v1.0's notice plus an appended # permissive grant, so the v2.0 rule (anchored on that addition) must precede # the v1.0 base-notice fallback. Run by parse_tu_berlin after the TuBerlin gate. # The trailing ".0" is stripped (TU-Berlin-1.0 -> TU-Berlin-1). TU_BERLIN_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile(r"we append this additional permission", rex.IGNORECASE), "TU-Berlin-2.0", ), ReSub( rex.compile( r"Any use of this software is permitted provided that this notice " r"is not removed and that neither the authors nor the Technische " r"Universitaet Berlin", rex.IGNORECASE, ), "TU-Berlin-1.0", ), ) # Mulan Permissive Software License (Chinese text) - the two versions differ by # the "第1版"/"第2版" (v1/v2) marker in the grant clause. Run by parse_mulan after # the MulanPsl gate. The trailing ".0" is stripped (MulanPSL-1.0 -> MulanPSL-1). MULAN_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile(r"修改及分发受木兰宽松许可证 第1版", rex.IGNORECASE), "MulanPSL-1.0", ), ReSub( rex.compile(r"修改及分发受木兰宽松许可证 第2版", rex.IGNORECASE), "MulanPSL-2.0", ), # English standardLicenseHeader application notice (grant by reference). ReSub( rex.compile( r"licensed under (?:the )?Mulan PSL v1\. You can use this software " r"according to the terms and conditions of the Mulan PSL v1", rex.IGNORECASE, ), "MulanPSL-1.0", ), ReSub( rex.compile( r"licensed under (?:the )?Mulan PSL v2\. You can use this software " r"according to the terms and conditions of the Mulan PSL v2", rex.IGNORECASE, ), "MulanPSL-2.0", ), ) # OSL-template derivatives (Rosen's Open Software License boilerplate reused # under a new name). Run by parse_osl_derived after the OslDerived gate; each # rule pairs the licence's own title-and-version with the shared "applies to any # original work of authorship" clause, so a bare name reference declines. The # trailing ".0" is stripped (TGPPL-1.0 -> TGPPL-1, UCL-1.0 -> UCL-1). OSL_DERIVED_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"Transitive Grace Period Public Licence \( ?TGPPL ?\) v\. 1\.0 " r"This Transitive Grace Period Public Licence \(the License ?\) " r"applies to any original work of authorship", rex.IGNORECASE, ), "TGPPL-1.0", ), ReSub( rex.compile( r"Upstream Compatibility License v\. 1\.0 \(UCL-1\.0\) This Upstream " r"Compatibility License \(the License ?\) applies to any original " r"work of authorship", rex.IGNORECASE, ), "UCL-1.0", ), # standardLicenseHeader application notices (grant by reference). ReSub( rex.compile( r"Licensed under the Transitive Grace Period Public Licence " r"version 1\.0", rex.IGNORECASE, ), "TGPPL-1.0", ), ReSub( rex.compile( r"Licensed under the Upstream Compatibility License 1\.0", rex.IGNORECASE, ), "UCL-1.0", ), ) # Microsoft Use of Data Agreements - the Computational (C-UDA) and Open (O-UDA) # variants share a template; run by parse_uda after the UdaData gate. The # trailing ".0" is stripped (C-UDA-1.0 -> C-UDA-1, O-UDA-1.0 -> O-UDA-1). UDA_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"Computational Use of Data Agreement, Version 1\.0 \(the C-UDA", rex.IGNORECASE, ), "C-UDA-1.0", ), ReSub( rex.compile( r"Open Use of Data Agreement, Version 1\.0 \(the O-UDA", rex.IGNORECASE, ), "O-UDA-1.0", ), ) # ASWF Digital Assets License - the two versions differ only by the title # version line. Run by parse_aswf after the AswfAssets gate (which fires on the # distinctive "digital assets" redistribution clause). The trailing ".0" is # stripped (ASWF-Digital-Assets-1.0 -> ASWF-Digital-Assets-1). ASWF_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile(r"ASWF Digital Assets License v1\.1", rex.IGNORECASE), "ASWF-Digital-Assets-1.1", ), ReSub( rex.compile(r"ASWF Digital Assets License v1\.0", rex.IGNORECASE), "ASWF-Digital-Assets-1.0", ), ) # MITRE terms-of-use grants (CAPEC, CVE) - both authorise reuse provided the # MITRE copyright designation is reproduced. Run by parse_mitre after the # MitreTou gate; each rule names the dataset granted. MITRE_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"license to use Common Attack Pattern Enumeration and " r"Classification \(CAPEC", rex.IGNORECASE, ), "CAPEC-tou", ), ReSub( # Anchor on the grant verb, not a bare "CVE" mention: the short form # collides with generic CVE references in source/docs (verified against # codesearch.debian.net), while this grant clause is unique to the licence. rex.compile( r"and distribute Common Vulnerabilities and Exposures \(CVE", rex.IGNORECASE, ), "cve-tou", ), ) # CMU/Mach permission notice (Carnegie Mellon, reused by NRL) - both share the # "ALLOWS FREE USE ... IN ITS AS IS CONDITION" disclaimer that fires the CmuMach # gate; run by parse_cmu_mach. The documentation variant carries the extra # "supporting documentation" clause; the bare gate disclaimer is the no-doc form. CMU_MACH_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile(r"appear in supporting documentation", rex.IGNORECASE), "CMU-Mach", ), ReSub( rex.compile( r"ALLOWS FREE USE OF THIS SOFTWARE IN ITS AS IS CONDITION", rex.IGNORECASE, ), "CMU-Mach-nodoc", ), ) # Tatu Ylonen's original SSH permission notice - both forms share the "the code I # have written ... can be used freely" grant that fires the Ssh gate; run by # parse_ssh. The full OpenSSH notice adds the "legal status ... some combination" # paragraph; the short form is just the grant clause. SSH_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"The legal status of this program is some combination of all these", rex.IGNORECASE, ), "SSH-OpenSSH", ), ReSub( rex.compile( r"the code I have written for this software can be used freely " r"for any purpose", rex.IGNORECASE, ), "SSH-short", ), ) # Boehm GC permission notice - both forms share the "a notice that the code was # modified is included" clause that fires the BoehmGc gate; run by parse_boehm. # The without-fee variant grants "without fee"; the base form omits it. BOEHM_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"use or copy this software for any purpose is hereby granted " r"without fee", rex.IGNORECASE, ), "Boehm-GC-without-fee", ), ReSub( rex.compile( r"granted to use or copy this program for any purpose", rex.IGNORECASE ), "Boehm-GC", ), ) CECILL_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"Une liste de questions fréquemment posées se " r"trouve sur le site web officiel de la famille " r"des licences CeCILL " r"\(https?[: ]/?/?www.cecill.info/index.fr.html\) " r"pour toute clarification qui serait nécessaire\.", rex.IGNORECASE, ), "CeCILL-2.1", ), ReSub( rex.compile( r"Frequently asked questions can be found on the " r"official website of the CeCILL licenses family " r"\(https?[: ]/?/?www.cecill.info/index.en.html\)" r"for any necessary clarification\. ", rex.IGNORECASE, ), "CeCILL-2.1", ), ReSub( rex.compile(r"un modèle de diffusion en logiciel libre", rex.IGNORECASE), "CeCILL-2", ), ReSub( rex.compile( r"The exercising of these rights is conditional " r"upon certain obligations for users so as to preserve " r"this status for all subsequent redistributions\.", rex.IGNORECASE, ), "CeCILL-2", ), ReSub( rex.compile( r"As a counterpart to the access to the source " r"code and rights to copy, modify and redistribute " r"granted by the license, users are provided only " r"with a limited warranty and the software\'s author, " r"the holder of the economic rights, and the successive " r"licensors only have limited liability\.", rex.IGNORECASE, ), "CeCILL-1.1", ), ReSub( rex.compile(r"un modèle de diffusion (?:« ?)?open source ?»?", rex.IGNORECASE), "CeCILL-1", ), ReSub( rex.compile( r"Nevertheless, access to the source code, and " r"the resulting rights to copy, modify and redistribute " r"only provide users with a limited warranty and the " r"software\'s author, the holder of the economic " r"rights, and the successive licensors only have " r"limited liability\. ", rex.IGNORECASE, ), "CeCILL-1", ), ) CNRI_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"Licensee may not use CNRI trademarks or trade " r"name, including [^ ]+ or CNRI, in a trademark " r"sense to endorse or promote products or services " r"of Licensee, or any third party\. Licensee may " r"use the mark [^ ]+ in connection with Licensee\'s " r"derivative versions that are based on or incorporate " r"the Software, but only in the form ", rex.IGNORECASE, ), "CNRI-Jython", ), ReSub( rex.compile( r"URL:? ?https?[: ]/?/?hdl\.handle\.net/" r"1895.22/1006", rex.IGNORECASE, ), "CNRI-Jython", ), ReSub( rex.compile( r"Notwithstanding the foregoing, with regard to " r"derivative works based on " r".+" r"that incorporate non[- ]separable material that " r"was previously distributed under the " r"GNU General Public License", rex.IGNORECASE, ), "CNRI-Python-GPL-Compatible", ), ReSub( rex.compile( r"URL:? ?https?[: ]/?/?hdl\.handle\.net/" r"1895.22/1013", rex.IGNORECASE, ), "CNRI-Python-GPL-Compatible", ), ReSub( rex.compile( r"URL:? ?https?[: ]/?/?hdl\.handle\.net/" r"1895.22/1011", rex.IGNORECASE, ), "CNRI-Python", ), ) EFL_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"If the binary program depends on a modified version " r"of this package, you are encouraged to publicly " r"release the modified version of this package", rex.IGNORECASE, ), "EFL-2", ), ReSub( rex.compile( r"if the binary program depends on a modified version " r"of this package, you must publicly release the " r"modified version of this package", rex.IGNORECASE, ), "EFL-1", ), ) GNU_VERSION_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"version ([\d.]+) (?:of the License,? )?or " r"(?:\(? ?at your option ?\)? )?" r"version ([\d.]+),? or,? (?:(?:any )?later versions? " r"(?:accepted|approved) by the " r"membership of KDE ?e\.?V\.?|at the discretion of KDE ?e\.?V\.? " r"(?:[^ ]+ ){,20}any later version)", rex.IGNORECASE, ), r"\1+\2+KDEeV", ), ReSub( rex.compile( r"General Public License version ([\d.]+) (?:and|or),? " r"(?:\(? ?at your option ?\)? )?version " r"([\d.]+) as published by the Free Software Foundation", rex.IGNORECASE, ), r"\1+\2", ), ReSub( rex.compile( r"General Public License as published by the Free Software " r"Foundation[,;]? (?:either )?version ([\d.]+) " r"(?:of the License,? )?" r"(?:and|or),? (?:\(? ?at your option ?\)? )?(?:version )?" r"([\d.]+?)[.,;]? ", rex.IGNORECASE, ), r"\1+\2", ), ReSub( rex.compile( r"version ([\d.]+) of the License,? or,? " r"(?:\(? ?at your option ?\)? )?" r"(?:any later version accepted by the " r"membership of KDE e.V.|at the discretion of KDE ?e\.?V\.? " r"(?:[^ ]+ ){,20}any later version)", rex.IGNORECASE, ), r"\1+KDEeV", ), ReSub( rex.compile( r"either version ([^ ]+?)(?: of the License)?[;,] " r"or[ ,(]+at your option[ ,)]+any later version", rex.IGNORECASE, ), r"\1+", ), ReSub( rex.compile( r"(?:modify it|used) under the terms of the GNU (?:Affero )?" r"(?:Lesser(?:/Library)? |Library(?:/Lesser)? )?General Public " r"Licen[cs]e (?:as published by the Free Software Foundation[;,]? )?" r"\(?(?:either )?version ?([^ ]+?)[;,)]* (?:as published by the Free " r"Software Foundation[;,]? )?(?:of the Licen[sc]e,? )?or " r"(?:later|(?:\(? ?at your option ?\)?,? )?any later version)", rex.IGNORECASE, ), r"\1+", ), ReSub( rex.compile( r"licensed under (?:[^ ]+ ){,10}[AL]?GPL,? ?(?:version|v)" r" ?([\d.]+) (?:(?:or )?(?:higher|later)|or[, (]+at your " r"option[, )]+any later version)", rex.IGNORECASE, ), r"\1+", ), ReSub( rex.compile( r"General Public License,? Version ([\d.]+) or later", rex.IGNORECASE, ), r"\1+", ), ReSub( rex.compile( r"under the terms of (?:the )?version ([^ ]+) or later of the " r"(?:GNU|[AL]?GPL)", rex.IGNORECASE, ), r"\1+", ), ReSub( rex.compile( r"GNU (?:Affero )?(?:Lesser(?:/Library)? |Library )?" r"General Public License (?:as )?published by the Free " r"Software Foundation[;,] either version ([^ ]+?)[.,]? " r"(?:of the License )?or " r"(?:\(at your option\) )?any later version", rex.IGNORECASE, ), r"\1+", ), ReSub( rex.compile( r"licensed under (?:[^ ]+ ){,10}[AL]?GPL ?(?:version|v)" r" ?([\d.]+?)[.,]? ", rex.IGNORECASE, ), r"\1", ), ReSub( rex.compile( r"(?:used|modify it) under the terms (?:and conditions )?" r"of the GNU " r"(?:Affero )?" r"(?:Lesser(?:/Library)? |Library )?General Public " r"License,? (?:as published by the Free Software " r"Foundation;? )?(?:either )?" r"Version ([\d.]+?)(?:[.,;]|as)? ", rex.IGNORECASE, ), r"\1", ), ReSub( rex.compile( r"GNU (?:Affero )?(?:Lesser(?:/Library)? |Library )?" r"General Public License, version ([^ ]+?)[ .,;]", rex.IGNORECASE, ), r"\1", ), ReSub( rex.compile( r"version ([\d.]+?)[.,]? (?:\(?only\)?.? )?" r"(?:of the GNU (?:Affero )?" r"(?:Lesser(?:/Library)? |Library )?" r"General Public License )?(?:as )?published by the " r"Free Software Foundation", rex.IGNORECASE, ), r"\1", ), ReSub( rex.compile( r"GNU (?:Affero )?(?:Lesser(?:/Library)? |Library )?" r"General Public License (?:as )?published by the Free " r"Software Foundation[;,] version ([^ ]+?)[.,]? ", rex.IGNORECASE, ), r"\1", ), ReSub( rex.compile( r"(?:General Public License|GPL) (?:Vers.?ion )?([\d.]+?)[.,]? ", rex.IGNORECASE, ), r"\1", ), ) GNU_EXCEPTION_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile( r"As a special exception, if you create a document which uses " r"this font, and embed this font or unaltered portions of this " r"font into the document, this font does not by itself cause " r"the resulting document to be covered by the GNU General " r"Public License. This exception does not however invalidate " r"any other reasons why the document might be covered by the " r"GNU General Public License. If you modify this font, you may " r"extend this exception to your version of the font, but you " r"are not obligated to do so. If you do not wish to do so, " r"delete this exception statement from your version.", rex.IGNORECASE, ), r"Font", ), ReSubGroup( ( ReSub( rex.compile( r"under certain conditions as described in " r"each individual source file", rex.IGNORECASE, ), r"OpenSSL-cryptsetup", ), ReSub( rex.compile( r"link the code of portions of this program with the OpenSSL" r"\s+library\.\s+You must obey", rex.IGNORECASE, ), r"OpenSSL-sqlitestudio", ), ReSub( rex.compile( r"combining OpenSSL with KVIrc", rex.IGNORECASE, ), r"OpenSSL-kvirc", ), ReSub( rex.compile( r"permission for vsftpd to be linked to the OpenSSL", rex.IGNORECASE, ), r"OpenSSL-vsftpd", ), ReSub( rex.compile( r"permission to link the code of its release " r"of x11vnc with the OpenSSL", rex.IGNORECASE, ), r"OpenSSL-x11vnc", ), ReSub( rex.compile( r"Special exception for linking OpenVPN with OpenSSL", rex.IGNORECASE, ), r"openvpn-openssl", ), ReSub( rex.compile( r"permission to dynamically link rsync with the OpenSSL", rex.IGNORECASE, ), r"rsync-linking", ), ReSub( rex.compile( r"Additional permission under GPLv3 section 7" r".{1,400}" r"linking or combining it with OpenSSL", rex.IGNORECASE | rex.DOTALL, ), r"GPL-3-389-ds", ), ReSub( rex.compile( r"As a special exception, the copyright holders give permission " r"to link the code of portions of this program with the OpenSSL " r"library", rex.IGNORECASE, ), r"OpenSSL", ), ) ), ReSubGroup( ( ReSub( rex.compile( r"(?:" r"AUTOCONF CONFIGURE SCRIPT EXCEPTION" r"|" r"configuration script generated by Autoconf" r".{1,250}" r"section 7 of the GNU General Public License, version 3" r")", rex.IGNORECASE | rex.DOTALL, ), r"AutoConf-3", ), ReSub( rex.compile( r"the respective Autoconf Macro[\' ]?s copyright owner " r"gives unlimited permission", rex.IGNORECASE, ), r"AutoConf-macro", ), ReSub( rex.compile( r"(?:" r"As a special exception to the GNU General Public License, " r"if you distribute this file as part of a program that " r"contains a configuration script generated by Autoconf" r"|" r"the Free Software Foundation gives unlimited permission to " r"copy, distribute and modify the configure scripts that are " r"the output of Autoconf\." r")", rex.IGNORECASE, ), r"AutoConf", ), ) ), ReSubGroup( ( ReSub( rex.compile( r"when this file is copied by Bison into a Bison output " r"file, you may use that output file without restriction", rex.IGNORECASE, ), r"Bison-1.24", ), ReSub( rex.compile( r"As a special exception, you may create a larger work that " r"contains part or all of the Bison parser skeleton and " r"distribute that work under terms of your choice", rex.IGNORECASE, ), r"Bison", ), ) ), ReSubGroup( ( ReSub( rex.compile( r"with the following modification:\s*" r"As a special exception, the copyright holders of this " r"library give you permission to link this library with " r"independent modules to produce an executable", rex.IGNORECASE | rex.DOTALL, ), r"Independent-modules", ), ReSub( rex.compile( r"As a special exception, the copyright holders of this " r"library give you permission to link this library with " r"independent modules to produce an executable, regardless " r"of the license terms of these independent modules, and to " r"copy and distribute the resulting executable under terms " r"of your choice, provided that you also meet, for each " r"linked independent module, the terms and conditions of " r"the license of that module\.", rex.IGNORECASE, ), r"ClassPath", ), ) ), ReSubGroup( ( ReSub( rex.compile( r"(?:" r"GCC RUNTIME LIBRARY EXCEPTION" r"|" r"This GCC Runtime Library Exception" r".{1,100}" r"section 7" r")", rex.IGNORECASE | rex.DOTALL, ), r"GCC-3.1", ), ReSub( rex.compile( r"In addition to the permissions in the GNU (?:Lesser )?" r"General Public License, the Free Software Foundation " r"gives you unlimited permission to link the compiled " r"version of this file", rex.IGNORECASE, ), r"GCC", ), ) ), ReSub( rex.compile( r"As a special exception to the (?:[^ ]+ ){,5}if you " r"distribute this file as part of a program or library that " r"is built using GNU Libtool, you may include this file under " r"the same distribution terms that you use for the rest of that " r"program", rex.IGNORECASE, ), r"LibTool", ), ReSub( rex.compile( r"In addition, as a special exception, (?:[^\s]+ ){,8}" r"gives You the additional right to link the code of " r"this Program with code not covered under the GNU " r'General Public License \(?[" ]?Non[- ]?GPL Code[" ]?\)? and ' r"to distribute linked combinations including the " r"two, subject to the limitations in this paragraph. " r"Non-GPL Code permitted under this exception must only " r"link to the code of this Program through those well " r"defined interfaces identified in the file named EXCEPTION " r'found in the source code files \(?the "?Approved ' r'Interfaces[" ]?[\) ]?\. The files of Non[- ]?GPL Code may ' r"instantiate templates or use macros or inline functions " r"from the Approved Interfaces without causing the resulting " r"work to be covered by the GNU General Public License. Only " r"(?:[^\s]+ ){,8}may make changes or additions to the list of " r"Approved Interfaces. You must obey the GNU General Public " r"License in all respects for all of the Program code and " r"other code used in conjunction with the Program except the " r"Non-GPL Code covered by this exception. If you modify this " r"file, you may extend this exception to your version of the " r"file, but you are not obligated to do so. If you do not " r"wish to provide this exception without modification, you " r"must delete this exception statement from your version and " r"license this file solely under the GPL without exception.", rex.IGNORECASE, ), r"389", ), ReSub( rex.compile( r"This copyright does NOT cover user programs that " r"run in CLISP and third-party packages not part " r"of CLISP, if a", rex.IGNORECASE, ), r"CLISP", ), ReSub( rex.compile( r'A FOSS application developer \([“" ]?you[”" ]? or ' r'[“" ]?your[”" ]?\) may distribute a Derivative ' r"Work provided that you and the Derivative Work " r"meet all of the following conditions", rex.IGNORECASE, ), r"DigiRule-FOSS", ), ReSub( rex.compile( r"As a special exception, if other files instantiate " r"templates or use macros or inline functions from " r"this file, or you compile this file and link " r"it with other works to produce a work based on " r"this file, this file does not by itself cause " r"the resulting work to be covered by the GNU General " r"Public License\. However the source code for " r"this file must still be made available in accordance " r"with section \(3\) of the GNU General Public License\.", rex.IGNORECASE, ), r"eCos-2", ), ReSub( rex.compile( r"The FLTK library and included programs are provided " r"under the terms of the GNU Library General Public " r"License \(LGPL\) with the following exceptions", rex.IGNORECASE, ), r"FLTK", ), ReSub( rex.compile( r"As a special exception, the copyright holder of " r"FreeRTOS gives you permission to link FreeRTOS " r"with independent modules that communicate with " r"FreeRTOS solely through the FreeRTOS API interface, " r"regardless of the license terms of these independent " r"modules, and to copy and distribute the resulting " r"combined work under terms of your choice, provided " r"that " r"Every copy of the combined work is accompanied by " r"a written statement that details to the recipient " r"the version of FreeRTOS used and an offer by yourself " r"to provide the FreeRTOS source code \(including any " r"modifications you may have made\) should the " r"recipient request it.", rex.IGNORECASE, ), r"FreeRTOS", ), ReSub( rex.compile( r"As a special exception, if you link this library " r"with other files to produce an executable, this " r"library does not by itself cause the resulting " r"executable to be covered by the GNU General Public " r"License.", rex.IGNORECASE, ), r"GNU-JavaMail", ), ReSub( rex.compile( r"In addition, as a special exception, .+" r"gives permission to link the code of this program " r"with the proprietary Java implementation provided " r"by Sun \(or other vendors as well\), and distribute " r"linked combinations including the two.", rex.IGNORECASE, ), r"i2p-Java", ), ReSub( rex.compile(r"Special exception for LZMA compression module", rex.IGNORECASE), r"LZMA", ), ReSub( rex.compile( r"if other files instantiate templates " r"or use macros or inline functions from this file, " r"or you compile this file and link it with other " r"files to produce an executable, this file does " r"not by itself cause the resulting executable " r"to be covered by the GNU General Public License.", rex.IGNORECASE, ), r"MIF", ), ReSub( rex.compile( r"Open CASCADE Exception \(version ([\d.]+)\) " r"to GNU ", rex.IGNORECASE, ), r"OCCT-\1", ), ReSub( rex.compile( r"unlimited permission to link or embed compiled bootloader" r" and related files into combinations with other programs", rex.IGNORECASE, ), r"Bootloader", ), ReSub( rex.compile( r"use specified GPL-licensed RRDtool", rex.IGNORECASE, ), r"RRDtool-FLOSS", ), ReSubGroup( ( ReSub( rex.compile(r"The Qt Company Qt LGPL Exception", rex.IGNORECASE), r"Qt-LGPL-1.1", ), ReSub( rex.compile(r"Digia Qt LGPL Exception", rex.IGNORECASE), r"Digia-Qt", ), ReSub( rex.compile( r"Qt LGPL Exception version ([\d.]+) " r"As an additional permission to the GNU ", rex.IGNORECASE, ), r"Qt\1", ), ) ), ReSub( rex.compile( r"U-Boot License Exception:? " r"Even though ", rex.IGNORECASE, ), r"u-boot", ), ReSub( rex.compile( r"EXCEPTION NOTICE " r".{,4}As a special exception, the copyright holders " r"of this library give permission for additional " r"uses of the text contained in this release of " r"the library as licenced under the wxWindows " r"Library Licen[cs]e, applying either version 3.1 " r"of the Licen[cs]e, or \(at your option\) any later " r"version of the Licen[cs]e as published by the " r"copyright holders of version 3.1 of the Licen[cs]e " r"document\.", rex.IGNORECASE, ), r"WxWindows-3.1+", ), ReSub( rex.compile( r"If you modify this.{1,100}runtime libraries of Erlang/OTP", rex.IGNORECASE | rex.DOTALL, ), r"erlang-otp", ), ReSub( rex.compile( r"the Harbour Project gives permission for", rex.IGNORECASE, ), r"harbour", ), ReSub( rex.compile( r"Additional permission under the GNU Affero GPL version 3 " r"section 7", rex.IGNORECASE, ), r"romic", ), ReSub( rex.compile( r"Linking stunnel statically or dynamically with other modules", rex.IGNORECASE, ), r"stunnel", ), ReSub( rex.compile( r"if you link this library with other files,\s+" r"compiled with a Free Software compiler", rex.IGNORECASE | rex.DOTALL, ), r"SWI", ), ReSub( rex.compile( r"when this file is read by TeX", rex.IGNORECASE, ), r"Texinfo", ), ReSub( rex.compile( r"UNMODIFIED BINARY DISTRIBUTION LICENCE", rex.IGNORECASE, ), r"UBDL", ), ReSub( rex.compile( r"link this library with the CGAL library", rex.IGNORECASE, ), r"CGAL-linking", ), ReSub( rex.compile( r"work that uses the OCaml Core System", rex.IGNORECASE, ), r"OCaml-LGPL", ), ReSub( rex.compile( r"libpri may also be linked to the OpenH323 library", rex.IGNORECASE, ), r"libpri-OpenH323", ), ReSub( rex.compile( r"relaxation of clause 6 of the LGPL", rex.IGNORECASE, ), r"polyparse", ), ReSub( rex.compile( r"Qwt (?:library|License)", rex.IGNORECASE, ), r"Qwt-1", ), ReSub( rex.compile( r"Simple Library Usage License", rex.IGNORECASE, ), r"Simple-Library-Usage", ), ReSub( rex.compile( r"The Qt Company GPL Exception", rex.IGNORECASE, ), r"Qt-GPL-1", ), ReSub( rex.compile( r"As a special exception, if other files instantiate " r"generics from this", rex.IGNORECASE, ), r"GNAT", ), ReSub( rex.compile( r"compiled with a GNU compiler to produce an executable", rex.IGNORECASE, ), r"GNU-compiler", ), ReSub( rex.compile( r"GPL Cooperation Commitment", rex.IGNORECASE, ), r"GPL-CC", ), ReSub( rex.compile( r"non-gpl compatible GStreamer plugins", rex.IGNORECASE, ), r"GStreamer", ), ReSub( rex.compile( r"copyright holders of Gmsh give you permission to combine Gmsh", rex.IGNORECASE, ), r"Gmsh", ), ReSub( rex.compile( r"this copyright does (?:not|\*not\*) cover user programs that " r"use kernel\s+services by normal system calls", rex.IGNORECASE, ), r"Linux-syscall", ), ReSubGroup( ( ReSub( rex.compile( r"Additional permission under GNU GPL version 3 section 7" r".{1,400}" r"Corresponding Source for a non-source form", rex.IGNORECASE | rex.DOTALL, ), r"GPL-3-linking-source", ), ReSub( rex.compile( r"Additional permission under GNU GPL version 3 section 7", rex.IGNORECASE, ), r"GPL-3-linking", ), ) ), ReSub( rex.compile( r"released under the GNU LGPL and with independent modules " r"that communicate with", rex.IGNORECASE, ), r"GPL-3-interface", ), ReSub( rex.compile( r"authors of sane-airscan give permission", rex.IGNORECASE, ), r"SANE", ), ReSub( rex.compile( r"openjdk\.java\.net.*?with the following " r"clarification and special\s+exception", rex.IGNORECASE | rex.DOTALL, ), r"OpenJDK-assembly", ), ReSub( rex.compile( r"permission is granted to include these " r"font\s+programs in a Postscript or PDF file", rex.IGNORECASE | rex.DOTALL, ), r"PS-or-PDF-font", ), ReSub( rex.compile( r"In addition, when this program is distributed " r"with Asterisk in any form that would qualify", rex.IGNORECASE, ), r"Asterisk", ), ReSub( rex.compile( r"Specific permission is also granted to link " r"Asterisk with OpenSSL, OpenH323", rex.IGNORECASE, ), r"Asterisk-linking", ), ReSub( rex.compile( r"special exception to the GNU Lesser General Public License version 3.*?" r"Combined Work that links statically or dynamically.*?" r"without providing any Minimal Corresponding", rex.IGNORECASE | rex.DOTALL, ), r"LGPL-3-linking", ), ReSub( rex.compile( r"we declare that, for the purpose applying the LGPL to the Library, " r"an executable that results from linking a work that uses the Library " r"with the Library is considered a work that uses the Library " r"and is therefore NOT covered by the LGPL", rex.IGNORECASE, ), r"LLGPL", ), ) UNIVERSAL_EXCEPTION_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile(r"The Universal FOSS Exception", rex.IGNORECASE), r"Universal-FOSS", ), ) APACHE_EXCEPTION_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile( r"As an exception.*?portions of this Software " r"are embedded into the binary product", rex.IGNORECASE | rex.DOTALL, ), r"Swift", ), ReSubGroup( ( ReSub( rex.compile( r"Solderpad Hardware Licen[cs]e\s+Version\s+2\.1", rex.IGNORECASE, ), r"SHL-2.1", ), ReSub( rex.compile( r"Solderpad Hardware Licen[cs]e.*?wraparound licen[cs]e", rex.IGNORECASE | rex.DOTALL, ), r"SHL-2", ), ) ), ReSub( rex.compile( r"LLVM Exceptions to the Apache 2\.0 License", rex.IGNORECASE, ), r"LLVM", ), ReSub( rex.compile( r"Mini-XML.*?Exceptions to the Apache 2\.0 License", rex.IGNORECASE | rex.DOTALL, ), r"mxml", ), ) MIT_EXCEPTION_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile( r"As an exception.*?portions\s+of this Software " r"are embedded into a machine-executable object form", rex.IGNORECASE | rex.DOTALL, ), r"fmt", ), ) CC_BY_SA_EXCEPTION_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile( r"As a special exception.*?permission to copy, " r"modify, and distribute the example code" r".*?without restriction", rex.IGNORECASE | rex.DOTALL, ), r"GNOME-examples", ), ReSub( rex.compile( r"waives article 3 of the license with respect to these designs", rex.IGNORECASE, ), r"KiCad-libraries", ), ) QPL_EXCEPTION_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile( r"special exception to the Q Public Licen[cs]e.*?" r"without any of the additional requirements listed in clause 6c", rex.IGNORECASE | rex.DOTALL, ), r"QPL-INRIA-2004", ), ) BSD_EXCEPTION_SUBS: tuple[ReSub | ReSubGroup, ...] = ( ReSub( rex.compile( r"EXEMPTION FOR BINARY LIBRARY-LIKE PACKAGES", rex.IGNORECASE, ), r"PCRE2", ), ) # Order matters: the matches are concatenated, so the no-invariants tag # precedes the version and the "+" suffix trails it (e.g. GFDL-NIV-1.2+). GFDL_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"with no Invariant Sections", rex.IGNORECASE, ), r"-NIV", ), ReSub( rex.compile( r"GNU Free Documentation License[,.; ]*(?:Version|v)? ?" r"(\d+(?:\.[\d]+)*)", rex.IGNORECASE, ), r"-\1", ), ReSub( rex.compile( r"or any later version", rex.IGNORECASE, ), r"+", ), ) LPPL_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"LaTeX Project Public License" r"(?: Distributed from CTAN archives in directory " r"macros/latex/base/lppl\.txt)?" # The titled form "...License LPPL Version 1.3a" carries the letter # suffix that the boilerplate "either version 1.3" reference drops. r"(?: \(?LPPL\)?)?" r"[,;]? (?:either )?(?:version|v) " r"?([\d.]+[a-z]?)[; ]", rex.IGNORECASE, ), r"-\1", ), ReSub( rex.compile( r"LaTeX Project Public License,? (?:[^ ]+ ){,16}or " r"(?:\(?at your option\)? )?any later version", rex.IGNORECASE, ), r"+", ), ) MIT_SUBS: tuple[ReSub, ...] = ( # HPND ("historical permission notice") variants. The HP forms differ only # in how many holders they name: the 1986 form keeps the notice "in all # copies", the 1989 form "in all source code copies" naming several holders. ReSub( rex.compile( r"this notice appears in all copies, and that the name of", rex.IGNORECASE, ), "MIT-like-HP-1986", ), ReSub( rex.compile( r"this notice appears in all source code copies, and that none of " r"the names", rex.IGNORECASE, ), "MIT-like-HP-1989", ), ReSub( rex.compile( r"distribute, and sell this documentation for any purpose is hereby " r"granted", rex.IGNORECASE, ), "MIT-like-HPND-doc-sell", ), ReSub( rex.compile( r"provided that this copyright and permissions notice appear in all " r"copies and derivatives", rex.IGNORECASE, ), "MIT-like-HPND-Kevlin-Henney", ), ReSub( rex.compile( r"this notice appears in all copies\. This file is distributed " r"WITHOUT ANY WARRANTY", rex.IGNORECASE, ), "MIT-like-HPND-merchantability-variant", ), ReSub( rex.compile( r"available with usual research terms with the aim of retain " r"credits of the software", rex.IGNORECASE, ), "MIT-like-HPND-INRIA-IMAG", ), ReSub( rex.compile( r"sell this software and its documentation for any purpose is " r"hereby granted without fee, provided that the above copyright " r"notice appear in all copies\. This software is provided", rex.IGNORECASE, ), "MIT-like-HPND-sell-regexpr", ), ReSub( rex.compile( r"provided that this permission notice appear in supporting " r"documentation\. This permission notice shall be included in all " r"copies or substantial portions", rex.IGNORECASE, ), "MIT-like-HPND-sell-MIT-disclaimer-xserver", ), # The base "sell" variant: the advertising clause ("name of … not be used # in advertising") is what separates it from the plain X11 "sell" grant, # which carries no such clause and stays the generic family name. ReSub( rex.compile( r"sell this software and its documentation for any purpose is " r"hereby granted without fee, provided that the above copyright " r"notice appears in all copies and that both that copyright notice " r"and this permission notice appear in supporting documentation, " r"and that the name of .{0,30} not be used in advertising", rex.IGNORECASE, ), "MIT-like-HPND-sell-variant", ), # The schema variant of the DocBook grant (cf. the stylesheet rule); the # "in perpetuity" + "schema" wording is unique to it. ReSub( rex.compile( r"schema and its accompanying documentation for any purpose and " r"without fee is hereby granted in perpetuity", rex.IGNORECASE, ), "MIT-like-DocBook-Schema", ), # Each licensed portion must carry a "copying is by permission of " # notice in its documentation - distinctive among the MIT/X11-style grants. ReSub( rex.compile( r"Notice must be given in supporting documentation that copying " r"distribution is by permission of", rex.IGNORECASE, ), "MIT-like-LOOP", ), ReSub( rex.compile( r"NONINFRINGEMENT OF THIRD PARTY RIGHTS\. IN NO EVENT SHALL THE " r"AUTHORS BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL", rex.IGNORECASE, ), "MIT-like-clips", ), ReSub( rex.compile( r"The above copyright notice and this permission notice shall " r"be included in all copies of the Software, its documentation " r"and marketing (?:&? )?publicity materials, and acknowledgment " r"shall be given in the documentation, materials and software " r"packages that this Software was used.", rex.IGNORECASE, ), "MIT-advertising", ), ReSub( rex.compile( r"Permission to use, copy, modify, and distribute this software " r"and its documentation for any purpose and without fee is " r"hereby granted, provided that the above copyright notice " r"appear in all copies and that both that the copyright notice " r"and this permission notice and warranty disclaimer appear in " r"supporting documentation, and that the name of the author not " r"be used in advertising or publicity pertaining to distribution " r"of the software without specific, written prior permission.", rex.IGNORECASE, ), "Expat-like-Highscore", ), ReSub( rex.compile( r"Except as contained in this notice, the name of a copyright " r"holder shall not be used in advertising or otherwise to " r"promote the sale, use or other dealings in this Software " r"without prior written authorization of the copyright holder.", rex.IGNORECASE, ), "MIT-like-icu", ), ReSub( rex.compile( r"Except as contained in this notice, the name of a copyright " r"holder shall not be used in advertising or otherwise to " r"promote the sale, use or other dealings in this Software " r"without prior written authorization of the copyright holder.", rex.IGNORECASE, ), "MIT-like-icu", ), ReSub( rex.compile( r"The above copyright notice and this permission notice shall " r"be included in all copies of the Software and its " r"Copyright notices. In addition publicly documented " r"acknowledgment must be given that this software has " r"been used if no source code of this software is " r"made available publicly. This includes acknowledgments " r"in either Copyright notices, Manuals, Publicity and " r"Marketing documents or any documentation provided " r"with any product containing this software. This " r"License does not apply to any software that links " r"to the libraries provided by this software \(?statically " r"or dynamically[\) ]?, but only to the software provided.", rex.IGNORECASE, ), "MIT-enna", ), ReSub( rex.compile( r"The above copyright notice and this permission " r"notice shall be included in all copies of the " r"Software and its documentation and acknowledgment " r"shall be given in the documentation and software " r"packages that this Software was used.", rex.IGNORECASE, ), "MIT-feh", ), ReSub( rex.compile( r"Distributions of all or part of the Software intended to be " r"used by the recipients as they would use the unmodified " r"Software, containing modifications that substantially alter, " r"remove, or disable functionality of the Software, " r"outside of the documented configuration mechanisms " r"provided by the Software, shall be modified such " r"that the Original Author[\' ]?s bug reporting " r"email addresses and urls are either replaced " r"with the contact information of the parties responsible " r"for the changes, or removed entirely.", rex.IGNORECASE, ), "MITNFA", ), ReSub( rex.compile( r"The above copyright notice and this permission notice shall " r"be included in all copies of the Software and its Copyright " r"notices. In addition publicly documented acknowledgment must " r"be given that this software has been used if no source code " r"of this software is made available publicly. Making the source " r"available publicly means including the source for this " r"software with the distribution, or a method to get this " r"software via some reasonable mechanism \(electronic transfer " r"via a network or media\) as well as making an offer to supply " r"the source on request. This Copyright notice serves as an " r"offer to supply the source on (?:on )?request as well. Instead " r"of this, supplying acknowledgments of use of this software in " r"either Copyright notices, Manuals, Publicity and Marketing " r"documents or any documentation provided with any product " r"containing this software. This License does not apply to any " r"software that links to the libraries provided by this " r"software \(statically or dynamically\), but only to the " r"software provided.", rex.IGNORECASE, ), "MIT-Imlib2", ), ReSub( rex.compile( r"The above copyright notice including the dates of first " r"publication and either this permission notice or a reference " r"to https?[: ]/?/?oss.sgi.com/projects/FreeB/ shall be " r"included in all copies or substantial portions of the " r"Software\.", rex.IGNORECASE, ), "SGI-B-2", ), ReSub( rex.compile( r"The above copyright notice including the dates of first " r"publication and either this permission notice or a reference " r"to https?[: ]/?/?oss.sgi.com/projects/FreeB/ shall be " r"included in all copies or substantial portions of the " r"Software\.", rex.IGNORECASE, ), "SGI-B-2", ), ReSub( rex.compile( r"Permission to use, copy, modify, and distribute this software " r"and its documentation for any purpose and without fee is " r"hereby granted, provided that the above copyright notice " r"appear in all copies and that both the copyright notice and " r"this permission notice and warranty disclaimer appear in " r"supporting documentation, and that the name of " r"(?:[^ ]+ ){,16}not be used in advertising or publicity " r"pertaining to distribution of the software without specific, " r"written prior permission\. " r"(?:[^ ]+ ){,4}disclaims all warranties with regard to this " r"software, including all implied warranties of merchantability " r"and fitness. In no event shall (?:[^ ]+ ){,4}be liable for " r"any special, indirect or consequential damages or any damages " r"whatsoever resulting from loss of use, data or profits, " r"whether in an action of contract, negligence or other " r"tortious action, arising out of or in connection with the use " r"or performance of this software\.", rex.IGNORECASE, ), "MIT-like-SMLNJ", ), ReSub( rex.compile( r"The end-user documentation included with the redistribution, " r'if any, must include the following acknowledgment:? "?This ' r"product includes software developed by The XFree86 Project, " r"Inc \(https?[: ]/?/?www.xfree86.org[/ ]?\) and its " r'contributors[" ]?, in the same place and form as other ' r"third-party acknowledgments\. Alternately, this acknowledgment " r"may appear in the software itself, in the same form and " r"location as other such third-party acknowledgments\. ", rex.IGNORECASE, ), "XFree86-1.1", ), ReSub( rex.compile( r"Permission to use, copy, modify, and distribute this software and its " r"documentation for any purpose and without fee is hereby granted, " r"provided that the above copyright notice appear in all copies and that " r"both that copyright notice and this permission notice appear in " r"supporting documentation\. No representations are made about the " r"suitability of this software for any purpose\. It is provided " r"[\"]?as is[\"]? without express or implied warranty\." ), "MIT-like-HPND-Netrek", ), ReSub( rex.compile( r"Historical Permission Notice and Disclaimer" r".{,500}without fee is hereby granted", rex.IGNORECASE, ), "MIT-like-HPND", ), ReSub( rex.compile( r"if the software is modified in a manner creating derivative " r"copyright rights, appropriate legends may be placed on the " r"derivative work", rex.IGNORECASE, ), "MIT-like-HPND-DEC", ), # HPND named variants: each anchors on a substantive clause unique to it, # so it is named at SPDX granularity instead of collapsing to plain MIT/X11. ReSub( rex.compile( r"the author disclaims all warranties with regard to this software\.", rex.IGNORECASE, ), "MIT-like-HPND-Markus-Kuhn", ), ReSub( rex.compile( r"Export of this software outside of the United States of America " r"may require an export license", rex.IGNORECASE, ), "MIT-like-HPND-UC-export-US", ), ReSub( rex.compile( r"distribute this documentation for any purpose", rex.IGNORECASE, ), "MIT-like-HPND-doc", ), ReSub( rex.compile( r"if you modify this software you must label your software as " r"modified software", rex.IGNORECASE, ), "MIT-like-HPND-export-US-modify", ), ReSub( rex.compile( r"WITHIN THAT CONSTRAINT.{0,1500}PROVIDED AS IS AND WITHOUT ANY " r"EXPRESS OR IMPLIED WARRANTIES", rex.IGNORECASE, ), "MIT-like-HPND-export2-US", ), ReSub( rex.compile( r"THIS SOFTWARE IS NOT DESIGNED FOR USE IN SAFETY CRITICAL SYSTEMS", rex.IGNORECASE, ), "MIT-like-HPND-sell-variant-critical-systems", ), ReSub( rex.compile( r"appear in supporting documentation\.\s+Disclaimer\s+" r"THE SOFTWARE IS PROVIDED AS IS", rex.IGNORECASE, ), "MIT-like-HPND-sell-variant-MIT-disclaimer", ), ReSub( rex.compile( r"dealings in the software\.\s+Permission to use, copy, modify, " r"distribute, and sell", rex.IGNORECASE, ), "MIT-like-HPND-sell-variant-MIT-disclaimer-rev", ), ReSub( rex.compile( r"original copyright notices appear in all copies.{0,400}" r"THE SOFTWARE IS PROVIDED AS IS , WITHOUT WARRANTY OF ANY KIND", rex.IGNORECASE, ), "MIT-like-HPND-MIT-disclaimer", ), # The variants below share their distinctive text with a sibling that has a # rule above, so they must stay after it: HPND-UC after HPND-UC-export-US, # and HPND-export-US after HPND-export-US-modify / HPND-export2-US. ReSub( rex.compile( r"appear in supporting documentation\. This software is provided " r"as is without express or implied warranty", rex.IGNORECASE, ), "MIT-like-HPND-Pbmplus", ), ReSub( rex.compile( r"notice be given in supporting documentation that copying and " r"distribution is by permission", rex.IGNORECASE, ), "MIT-like-HPND-Fenneberg-Livingston", ), ReSub( rex.compile( r"and without fee is hereby granted, provided that the above " r"copyright notice appear in all copies\.\s", rex.IGNORECASE, ), "MIT-like-HPND-UC", ), ReSub( rex.compile( r"WITHIN THAT CONSTRAINT.{0,800}makes no representations about the " r"suitability of this software for any purpose\. It is provided as is", rex.IGNORECASE, ), "MIT-like-HPND-export-US", ), # MIT/Expat-grant variants routed here via the bare MIT/X11 grant rules; each # anchors on a distinctive clause. MIT-0 also matches Clips, which is fine: # the Clips rule is first in this list and wins for the Clips text. ReSub( rex.compile( r"to permit persons to whom the Software is furnished to do so\. " r"THE SOFTWARE IS PROVIDED AS IS , WITHOUT WARRANTY OF ANY KIND", rex.IGNORECASE, ), "MIT-0", ), ReSub( rex.compile( r"MODIFICATIONS TO THIS FILE MAY MEAN IT NO LONGER ACCURATELY " r"REFLECTS", rex.IGNORECASE, ), "MIT-Khronos-old", ), ReSub( rex.compile( r"without license or royalty fees, to use, copy, modify, and " r"distribute this software and its documentation for any purpose, " r"provided that the above copyright notice and the following two " r"paragraphs", rex.IGNORECASE, ), "MIT-Modern-Variant", ), ReSub( rex.compile( r"asked to send the modifications to the original developer so " r"that they can be incorporated into the canonical version", rex.IGNORECASE, ), "MIT-STK", ), ReSub( rex.compile( r"appear in supporting documentation\. The above copyright notice " r"and this permission notice shall be included in all copies or " r"substantial portions", rex.IGNORECASE, ), "MIT-open-group", ), # More X11/MIT-style variants, each anchored on a distinctive clause. The # X11-* ids follow SPDX directly; the others use the MIT-like- form. ReSub( rex.compile( r"THE MATERIAL EMBODIED ON THIS SOFTWARE IS PROVIDED TO YOU AS-IS", rex.IGNORECASE, ), "MIT-like-SGI-OpenGL", ), ReSub( rex.compile( r"this permission and disclaimer notice appear in supporting " r"documentation", rex.IGNORECASE, ), "MIT-like-UMich-Merit", ), ReSub( rex.compile( r"REPRESENTATIONS ABOUT THE ACCURACY OR SUITABILITY OF THIS " r"MATERIAL", rex.IGNORECASE, ), "MIT-like-metamail", ), ReSub( rex.compile( r"no liability with respect to the infringement of copyrights, " r"trade secrets or any patents", rex.IGNORECASE, ), "MIT-like-xlock", ), ReSub( rex.compile( r"distribute with modifications, sublicense", rex.IGNORECASE, ), "X11-distribute-modifications-variant", ), ReSub( rex.compile( r"sell copies of the Software\. The above copyright notice and " r"this permission notice shall be included", rex.IGNORECASE, ), "X11-no-permit-persons", ), ReSub( rex.compile( r"shall be included in all copies or substantial portions of the " r"Software\. Except as contained in this notice", rex.IGNORECASE, ), "X11-swapped", ), ) ZPL_SUBS: tuple[ReSub, ...] = ( ReSub( rex.compile( r"(?:the )?Zope Public License(?: \(?ZPL\)?)?" r",? (?:Version|v)?[ ]?([\d.]+?)(?:[.0]*)[^\d.]", rex.IGNORECASE, ), r"-\1", ), ) decopy-0.3.2/decopy/licensing/license_texts.py000066400000000000000000000110301522176660400214640ustar00rootroot00000000000000#!/usr/bin/env python3 # Copyright: 2026, Maximiliano Curia # # License: ISC # Permission to use, copy, modify, and/or distribute this software for any # purpose with or without fee is hereby granted, provided that the above # copyright notice and this permission notice appear in all copies. # . # THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES WITH # REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY # AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR ANY SPECIAL, DIRECT, # INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM # LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE # OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR # PERFORMANCE OF THIS SOFTWARE. """Resolve a License stanza body for a detected licence name. Given a single stanza key (one licence, optionally a " with