efrotools package

Submodules

efrotools.android module

Functionality related to android builds.

class efrotools.android.GradleFilterSection(tag: str, firstline: int, lastline: int)

Bases: object

Filtered section of gradle file.

firstline: int

lastline: int

tag: str

efrotools.android.filter_gradle_file(buildfilename: str, enabled_tags: set[str]) → None

Filter ‘EFRO_IF’ sections in a gradle file.

efrotools.buildlock module

A system for sanity testing parallel build isolation.

class efrotools.buildlock.BuildLock(name: str, projroot: str)

Bases: object

Tries to ensure a build is not getting stomped on/etc.

efrotools.code module

Functionality for formatting, linting, etc. code.

efrotools.code.black_base_args(projroot: Path) → list[str]

Build base args for running black Python formatting.

efrotools.code.check_android_studio(projroot: Path, full: bool, verbose: bool) → None

Run Android Studio inspections on all our code.

efrotools.code.check_clioncode(projroot: Path, full: bool, verbose: bool) → None

Run clion inspections on all our code.

efrotools.code.check_cpplint(projroot: Path, full: bool) → None

Run cpplint on all our applicable code.

efrotools.code.check_pycharm(projroot: Path, full: bool, verbose: bool) → None

Run pycharm inspections on all our scripts.

efrotools.code.dmypy(projroot: Path) → None

Type check all of our scripts using mypy in daemon mode.

efrotools.code.format_cpp_str(projroot: Path, text: str, filename: str = 'untitled.cc') → str

Run clang-format inline on c++ code.

Note that some cpp formatting keys off the filename, so a fake one can be optionally provided.

efrotools.code.format_project_cpp_files(projroot: Path, full: bool) → None

Run clang-format on all of our source code (multithreaded).

efrotools.code.format_project_python_files(projroot: Path, full: bool) → None

Runs formatting on all of our Python code.

efrotools.code.format_python_str(projroot: Path | str, code: str) → str

Run our Python formatting on the provided inline code.

efrotools.code.get_code_filenames(projroot: Path, include_generated: bool) → list[str]

Return the list of files to lint-check or auto-format.

Be sure to pass False for include_generated if performing any operation that can modify files (such as formatting). Otherwise it could cause dirty generated files to not get updated properly when their sources change).

efrotools.code.get_script_filenames(projroot: Path) → list[str]

Return the Python filenames to lint-check or auto-format.

efrotools.code.mypy(projroot: Path, full: bool) → None

Type check all of our scripts using mypy.

efrotools.code.mypy_files(projroot: Path, filenames: list[str], full: bool = False, check: bool = True) → None

Run MyPy on provided filenames.

efrotools.code.pylint(projroot: Path, full: bool, fast: bool) → None

Run Pylint on all scripts in our project (with smart dep tracking).

efrotools.code.runpylint(projroot: Path, filenames: list[str]) → None

Run Pylint explicitly on files.

efrotools.code.sort_jetbrains_dict(original: str) → str

Given jetbrains dict contents, sort it the way jetbrains would.

efrotools.efrocache module

A simple cloud caching system for making built binaries & assets.

The basic idea here is the ballistica-internal project can flag file targets in its Makefiles as ‘cached’, and the public version of those Makefiles will be filtered to contain cache downloads in place of the original build commands. Cached files are gathered and uploaded as part of the pubsync process.

class efrotools.efrocache.CacheMetadata(executable: ~typing.Annotated[bool, <efro.dataclassio.IOAttrs object at 0x7ffa91ed5f70>])

Bases: object

Metadata stored with a cache file.

executable: IOAttrs object at 0x7ffa91ed6b70>]

efrotools.efrocache.filter_makefile(makefile_dir: str, contents: str) → str

Filter makefile contents to use efrocache lookups.

efrotools.efrocache.get_existing_file_hash(path: str) → str

Return the hash used for caching.

efrotools.efrocache.get_local_cache_dir() → str

Where we store local efrocache files we’ve downloaded.

Rebuilds will be able to access the local cache instead of re-downloading. By default each project has its own cache dir but this can be shared between projects by setting the EFROCACHE_DIR environment variable.

efrotools.efrocache.get_repository_base_url() → str

Return the base repository url (assumes cwd is project root).

efrotools.efrocache.get_target(path: str, batch: bool, clr: type[efro.terminal.ClrBase]) → str

Fetch a target path from the cache, downloading if need be.

efrotools.efrocache.update_cache(makefile_dirs: list[str]) → None

Given a list of directories containing Makefiles, update caches.

efrotools.efrocache.warm_start_cache(cachetype: str) → None

Run a pre-pass on the efrocache to improve efficiency.

This may fetch an initial cache archive, batch update mod times to reflect new cache maps, etc.

efrotools.emacs module

Stuff intended to be used from emacs

efrotools.emacs.py_examine(projroot: Path, filename: Path, line: int, column: int, selection: str | None, operation: str) → None

Given file position info, performs some code inspection.

efrotools.filecache module

Provides a system for caching linting/formatting operations.

class efrotools.filecache.FileCache(path: Path)

Bases: object

A cache of file hashes/etc. used in linting/formatting/etc.

get_dirty_files() → Sequence[str]

Return paths for all entries with no hash value.

mark_clean(files: Sequence[str]) → None

Marks provided files as up to date.

update(filenames: Sequence[str], extrahash: str) → None

Update the cache for the provided files and hash type.

Hashes will be checked for all files (incorporating extrahash) and mismatched hash values cleared. Entries for no-longer-existing files will be cleared as well.

write() → None

Writes the state back to its file.

efrotools.filecommand module

Operate on large sets of files efficiently.

efrotools.filecommand.file_batches(paths: list[str], batch_size: int = 1, file_filter: Callable[[str], bool] | None = None, include_mac_packages: bool = False) → Iterable[list[str]]

Efficiently yield batches of files to operate on.

Accepts a list of paths which can be files or directories to be recursed. The batch lists are buffered in a background thread so time-consuming synchronous operations on the returned batches will not slow the gather.

efrotools.genwrapper module

Functionality related to android builds.

efrotools.ios module

Tools related to ios development.

class efrotools.ios.Config(product_name: str, projectpath: str, scheme: str)

Bases: object

Configuration values for this project.

product_name: str

projectpath: str

scheme: str

class efrotools.ios.LocalConfig(sftp_host: str, sftp_dir: str)

Bases: object

Configuration values specific to the machine.

sftp_dir: str

sftp_host: str

efrotools.ios.push_ipa(root: Path, modename: str, signing_config: str | None) → None

Construct ios IPA and push it to staging server for device testing.

This takes some shortcuts to minimize turnaround time; It doesn’t recreate the ipa completely each run, uses rsync for speedy pushes to the staging server, etc. The use case for this is quick build iteration on a device that is not physically near the build machine.

efrotools.jsontools module

Json related tools functionality.

class efrotools.jsontools.NoIndent(value: Any)

Bases: object

Used to prevent indenting in our custom json encoder.

Wrap values in this before passing to encoder and all child values will be a single line in the json output.

class efrotools.jsontools.NoIndentEncoder(*args: Any, **kwargs: Any)

Bases: JSONEncoder

Our custom encoder implementing selective indentation.

default(o: Any) → Any

Implement this method in a subclass such that it returns a serializable object for o, or calls the base implementation (to raise a TypeError).

For example, to support arbitrary iterators, you could implement default like this:

def default(self, o):
    try:
        iterable = iter(o)
    except TypeError:
        pass
    else:
        return list(iterable)
    # Let the base class default method raise the TypeError
    return super().default(o)

encode(o: Any) → Any

Return a JSON string representation of a Python data structure.

>>> from json.encoder import JSONEncoder
>>> JSONEncoder().encode({"foo": ["bar", "baz"]})
'{"foo": ["bar", "baz"]}'

efrotools.lazybuild module

Functionality used for building.

class efrotools.lazybuild.LazyBuildContext(target: str, srcpaths: list[str], command: str, *, buildlockname: str | None = None, dirfilter: Callable[[str, str], bool] | None = None, filefilter: Callable[[str, str], bool] | None = None, srcpaths_fullclean: list[str] | None = None, srcpaths_exist: list[str] | None = None, manifest_file: str | None = None, command_fullclean: str | None = None)

Bases: object

Run a build if anything in some category is newer than a target.

This can be used as an optimization for build targets that always run. As an example, a target that spins up a VM and runs a build can be expensive even if the VM build process determines that nothing has changed and does no work. We can use this to examine a broad swath of source files and skip firing up the VM if nothing has changed. We can be overly broad in the sources we look at since the worst result of a false positive change is the VM spinning up and determining that no actual inputs have changed. We could recreate this mechanism purely in the Makefile, but large numbers of target sources can add significant overhead each time the Makefile is invoked; in our case the cost is only incurred when a build is triggered.

Note that target’s mod-time will always be updated to match the newest source regardless of whether the build itself was triggered.

run() → None

Do the thing.

efrotools.makefile module

Tools for parsing/filtering makefiles.

class efrotools.makefile.Makefile(contents: str)

Bases: object

Represents an entire Makefile.

find_assigns(name: str) → list[tuple[Section, int]]

Return section/index pairs for paragraphs containing an assign.

Note that the paragraph may contain other statements as well.

find_targets(name: str) → list[tuple[Section, int]]

Return section/index pairs for paragraphs containing a target.

Note that the paragraph may contain other statements as well.

get_output() → str

Generate a Makefile from the current state.

header_line_empty = '#                                                                              #'

header_line_full = '################################################################################'

class efrotools.makefile.Paragraph(contents: str)

Bases: object

Represents a continuous set of non-blank lines in a Makefile.

contents: str

get_logical_lines() → list[str]

Return contents broken into logical lines.

Lines joined by continuation chars are considered a single line.

class efrotools.makefile.Section(name: str | None, paragraphs: list[Paragraph])

Bases: object

Represents a section of a Makefile.

name: str | None

paragraphs: list[Paragraph]

efrotools.message module

Message related tools functionality.

efrotools.message.standard_message_receiver_gen_pcommand(*, projroot: Path, basename: str, source_module: str, is_async: bool, get_protocol_call: str = 'get_protocol', embedded: bool = False) → None

Used by pcommands generating efro.message receiver modules.

efrotools.message.standard_message_sender_gen_pcommand(*, projroot: Path, basename: str, source_module: str, enable_sync_sends: bool, enable_async_sends: bool, get_protocol_call: str = 'get_protocol', embedded: bool = False) → None

Used by pcommands taking a single filename argument.

efrotools.openalbuild module

Functionality to build the openal library.

efrotools.openalbuild.build_openal(arch: str, mode: str) → None

Do the thing.

efrotools.openalbuild.gather() → None

Gather the things. Assumes all have been built.

efrotools.pcommand module

Standard snippets that can be pulled into project pcommand scripts.

A snippet is a mini-program that directly takes input from stdin and does some focused task. This module is a repository of common snippets that can be imported into projects’ pcommand script for easy reuse.

efrotools.pcommand.clientprint(*args: Any, stderr: bool = False, end: str | None = None) → None

Print to client stdout.

Note that, in batch mode, the results of all clientprints will show up only after the command completes. In regular mode, clientprint() simply passes through to regular print().

efrotools.pcommand.clr() → type[ClrBase]

Like efro.terminal.Clr but for use with pcommand.clientprint().

This properly colorizes or doesn’t colorize based on whether the client where output will be displayed is running on a terminal. Regular print() output should still use efro.terminal.Clr for this purpose.

efrotools.pcommand.disallow_in_batch() → None

Utility call to raise a clean error if running under batch mode.

efrotools.pcommand.enter_batch_server_mode() → None

Called by pcommandserver when we start serving.

efrotools.pcommand.get_args() → list[str]

Return the args for the current pcommand.

efrotools.pcommand.is_batch() → bool

Is the current pcommand running under a batch server?

Commands that do things that are unsafe to do in server mode such as chdir should assert that this is not true.

efrotools.pcommand.pcommand_main(globs: dict[str, Any]) → None

Main entry point to pcommand scripts.

We simply look for all public functions in the provided module globals and call the one corresponding to the first passed arg.

efrotools.pcommand.run_client_pcommand(args: list[str], clrtp: type[ClrBase], logpath: str) → tuple[int, str, str]

Call a pcommand function as a server.

Returns a result code and stdout output.

efrotools.pcommandbatch module

Wrangles pcommandbatch; an efficient way to run small pcommands.

The whole purpose of pcommand is to be a lightweight way to run small snippets of Python to do bits of work in a project. The pcommand script tries to minimize imports and work done in order to keep runtime as short as possible. However, even an ‘empty’ pcommand still takes a fraction of a second due to the time needed to spin up Python and import a minimal set of modules. This can add up for large builds where hundreds or thousands of pcommands are being run.

To help fight that problem, pcommandbatch introduces a way to run pcommands by submitting requests to temporary local server daemons. This allows individual pcommand calls to go through a lightweight client binary that simply forwards the command to a running server. This cuts minimum pcommand runtimes down greatly. Building and managing the server and client are handled automatically, and systems which are unable to compile a client binary can fall back to using vanilla pcommand in those cases.

A few considerations must be made when using pcommandbatch. By default, all existing pcommands have been fitted with a disallow_in_batch() call which triggers an error under batch mode. These calls should be removed if/when each call is updated to work cleanly in batch mode. Guidelines for batch-friendly pcommands follow:

• Batch mode runs parallel pcommands in different background threads and may process thousands of commands in a single process. Batch-friendly pcommands must behave reasonably in such an environment.

• Batch-enabled pcommands must not call os.chdir() or sys.exit() or anything else having global effects. This should be self-explanatory considering the shared server model in use.

• Batch-enabled pcommands must not use environment-variables to influence their behavior. In batch mode this would unintuitively use the environment of the server and not of the client.

• Batch-enabled pcommands should not look at sys.argv. They should instead use pcommand.get_args(). Be aware that this value does not include the first two values from sys.argv (executable path and pcommand name) so is generally cleaner to use anyway. Also be aware that args are thread-local, so only call get_args() from the thread your pcommand was called in.

• Batch-enabled pcommands should not use efro.terminal.Clr for coloring terminal output; instead they should use pcommand.clr() which properly takes into account whether the client is running on a tty/etc.

• Standard print and log calls (as well as those of child processes) will wind up in the pcommandbatch server log and will not be seen by the user or capturable by the calling process. For batch-friendly printing, use pcommand.clientprint(). Note that, in batch mode, all output will be printed on the client after the command completes and stderr and stdout will be printed separately instead of intermingled. If a pcommand is long-running and prints at multiple times while doing its thing, it is probably not a good fit for batch-mode.

exception efrotools.pcommandbatch.IdleError

Bases: RuntimeError

Error we raise to quit peacefully.

class efrotools.pcommandbatch.Server(idle_timeout_secs: int, project_dir: str, instance: str, daemon: bool)

Bases: object

A server that handles requests from pcommandbatch clients.

run() → None

Do the thing.

efrotools.pcommandbatch.batchserver(idle_timeout_secs: int, project_dir: str, instance: str) → None

Run a server for handling batches of pcommands.

If a matching instance is already running, is a no-op.

efrotools.pcommandbatch.build_pcommandbatch(inpaths: list[str], outpath: str) → None

Create the binary or link regular pcommand.

efrotools.pcommands module

A set of lovely pcommands ready for use.

efrotools.pcommands.androidstudiocode() → None

Run Android Studio checks on our code.

efrotools.pcommands.check_clean_safety() → None

Ensure all files are are added to git or in gitignore.

Use to avoid losing work if we accidentally do a clean without adding something.

efrotools.pcommands.clioncode() → None

Run CLion checks on our code.

efrotools.pcommands.compile_collision_mesh_file() → None

Compile a mesh file.

efrotools.pcommands.compile_font_file() → None

Compile a font file.

efrotools.pcommands.compile_language_file() → None

Compile a language file.

efrotools.pcommands.compile_mesh_file() → None

Compile a mesh file.

efrotools.pcommands.compile_python_file() → None

Compile pyc files for packaging.

This creates hash-based PYC files in opt level 1 with hash checks defaulting to off, so we don’t have to worry about timestamps or loading speed hits due to hash checks. (see PEP 552). We just need to tell modders that they’ll need to clear these cache files out or turn on debugging mode if they want to tweak the built-in scripts directly (or go through the asset build system which properly recreates the .pyc files).

efrotools.pcommands.copy_python_file() → None

Copy Python files for packaging.

efrotools.pcommands.copy_win_extra_file() → None

Copy a windows extra file.

efrotools.pcommands.cpplint() → None

Run lint-checking on all code deemed lint-able.

efrotools.pcommands.dmypy() → None

Run mypy checks on our scripts using the mypy daemon.

efrotools.pcommands.echo() → None

Echo with support for efro.terminal.Clr args (RED, GRN, BLU, etc).

Prints a Clr.RST at the end so that can be omitted.

efrotools.pcommands.formatcode() → None

Format all of our C/C++/etc. code.

efrotools.pcommands.formatmakefile() → None

Format the main makefile.

efrotools.pcommands.formatscripts() → None

Format all of our Python/etc. code.

efrotools.pcommands.gen_empty_py_init() → None

Generate an empty __init__.py for a package dir.

Used as part of meta builds.

efrotools.pcommands.make_ensure() → None

Make sure a makefile target is up-to-date.

This can technically be done by simply make –question, but this has some extra bells and whistles such as printing some of the commands that would run. Can be useful to run after cloud-builds to ensure the local results consider themselves up-to-date.

efrotools.pcommands.make_target_debug() → None

Debug makefile src/target mod times given src and dst path.

Built to debug stubborn Makefile targets that insist on being rebuilt just after being built via a cloud target.

efrotools.pcommands.makefile_target_list() → None

Prints targets in a makefile.

Takes a single argument: a path to a Makefile.

efrotools.pcommands.mypy() → None

Run mypy checks on our scripts.

efrotools.pcommands.mypy_files() → None

Run mypy checks on provided filenames.

efrotools.pcommands.pycharm() → None

Run PyCharm checks on our scripts.

efrotools.pcommands.pylint() → None

Run pylint checks on our scripts.

efrotools.pcommands.pylint_files() → None

Run pylint checks on provided filenames.

efrotools.pcommands.pytest() → None

Run pytest with project environment set up properly.

efrotools.pcommands.pyver() → None

Prints the Python version used by this project.

efrotools.pcommands.requirements_upgrade() → None

Upgrade project requirements.

efrotools.pcommands.scriptfiles() → None

List project script files.

Pass -lines to use newlines as separators. The default is spaces.

efrotools.pcommands.spelling() → None

Add words to the PyCharm dictionary.

efrotools.pcommands.spelling_all() → None

Add all misspellings from a pycharm run.

efrotools.pcommands.sync() → None

Runs standard syncs between this project and others.

efrotools.pcommands.sync_all() → None

Runs full syncs between all efrotools projects.

This list is defined in the EFROTOOLS_SYNC_PROJECTS env var. This assumes that there is a ‘sync-full’ and ‘sync-list’ Makefile target under each project.

efrotools.pcommands.tool_config_install() → None

Install a tool config file (with some filtering).

efrotools.pcommands.try_repeat() → None

Run a command with repeat attempts on failure.

First arg is the number of retries; remaining args are the command.

efrotools.pcommands.tweak_empty_py_files() → None

Find any zero-length Python files and make them length 1.

efrotools.pcommands.urandom_pretty() → None

Spits out urandom bytes formatted for source files.

efrotools.pcommands.xcodebuild() → None

Run xcodebuild with added smarts.

efrotools.pcommands.xcoderun() → None

Run an xcode build in the terminal.

efrotools.pcommands.xcodeshow() → None

Open folder containing xcode build in the finder.

efrotools.pcommands2 module

Standard snippets that can be pulled into project pcommand scripts.

A snippet is a mini-program that directly takes input from stdin and does some focused task. This module is a repository of common snippets that can be imported into projects’ pcommand script for easy reuse.

efrotools.pcommands2.batchserver() → None

Run a server for handling pcommands.

efrotools.pcommands2.build_pcommandbatch() → None

Build a version of pcommand geared for large batches of commands.

efrotools.pcommands2.null() → None

Do nothing. Useful for speed tests and whatnot.

efrotools.pcommands2.openal_build_android() → None

Build openalsoft for android.

efrotools.pcommands2.openal_gather() → None

Gather built opealsoft libs into src.

efrotools.pcommands2.pcommandbatch_speed_test() → None

Test batch mode speeds.

efrotools.pcommands2.pyright() → None

Run Pyright checks on project Python code.

efrotools.pcommands2.sortlines() → None

Sort provided lines. For tidying import lists, etc.

efrotools.pcommands2.with_build_lock() → None

Run a shell command wrapped in a build-lock.

efrotools.project module

Project related functionality.

efrotools.project.get_non_public_legal_notice() → str

Return the one line legal notice we expect private repo files to have.

efrotools.project.get_non_public_legal_notice_prev() → str

Allows us to auto-update.

efrotools.project.get_public_legal_notice(style: Literal['python', 'c++', 'makefile', 'raw']) → str

Return the license notice as used for our public facing stuff.

‘style’ arg can be ‘python’, ‘c++’, or ‘makefile, or ‘raw’.

efrotools.project.getlocalconfig(projroot: Path | str) → dict[str, Any]

Return a project’s localconfig contents (or default if missing).

efrotools.project.getprojectconfig(projroot: Path | str) → dict[str, Any]

Return a project’s projectconfig contents (or default if missing).

efrotools.project.setprojectconfig(projroot: Path | str, config: dict[str, Any]) → None

Set the project config contents.

efrotools.pybuild module

Functionality related to building python for ios, android, etc.

efrotools.pybuild.android_patch() → None

Run necessary patches on an android archive before building.

efrotools.pybuild.android_patch_ssl() → None

Run necessary patches on an android ssl before building.

efrotools.pybuild.apple_patch(python_dir: str) → None

New test.

efrotools.pybuild.build_android(rootdir: str, arch: str, debug: bool = False) → None

Run a build for android with the given architecture.

(can be arm, arm64, x86, or x86_64)

efrotools.pybuild.build_apple(arch: str, debug: bool = False) → None

Run a build for the provided apple arch (mac, ios, or tvos).

efrotools.pybuild.gather(do_android: bool, do_apple: bool) → None

Gather per-platform python headers, libs, and modules into our src.

This assumes all embeddable py builds have been run successfully, and that PROJROOT is the cwd.

efrotools.pybuild.patch_modules_setup(python_dir: str, baseplatform: str) → None

Muck with the Setup.* files Python uses to build modules.

efrotools.pybuild.tweak_empty_py_files(dirpath: str) → None

Find any zero-length Python files and make them length 1

I’m finding that my jenkins server updates modtimes on all empty files when fetching updates regardless of whether anything has changed. This leads to a decent number of assets getting rebuilt when not necessary.

As a slightly-hacky-but-effective workaround I’m sticking a newline up in there.

efrotools.pybuild.winprune() → None

Prune unneeded files from windows python dists.

Should run this after dropping updated windows libs/dlls/etc into our src dirs.

efrotools.pylintplugins module

Plugins for pylint

efrotools.pylintplugins.failed_import_hook(modname: str) → None

Custom failed import callback.

efrotools.pylintplugins.func_annotations_filter(node: nc.NodeNG) → nc.NodeNG

Filter annotated function args/retvals.

This accounts for deferred evaluation available in in Python 3.7+ via ‘from __future__ import annotations’. In this case we don’t want Pylint to complain about missing symbols in annotations when they aren’t actually needed at runtime.

efrotools.pylintplugins.ignore_reveal_type_call(node: nc.NodeNG) → nc.NodeNG

Make ‘reveal_type()’ not trigger an error.

The ‘reveal_type()’ fake call is used for type debugging types with mypy and it is annoying having pylint errors pop up alongside the mypy info.

efrotools.pylintplugins.ignore_type_check_filter(if_node: nc.NodeNG) → nc.NodeNG

Ignore stuff under ‘if TYPE_CHECKING:’ block at module level.

efrotools.pylintplugins.register(linter: Any) → None

Unused here; we’re modifying the ast; not linters.

efrotools.pylintplugins.register_plugins(manager: astroid.Manager) → None

Apply our transforms to a given astroid manager object.

efrotools.pylintplugins.using_future_annotations(node: nc.NodeNG) → nc.NodeNG

Return whether postponed annotation evaluation is enabled (PEP 563).

efrotools.pylintplugins.var_annotations_filter(node: nc.NodeNG) → nc.NodeNG

Filter annotated function variable assigns.

This accounts for deferred evaluation.

efrotools.pyver module

This module defines the major Python version we are using in the project.

Tools that need to do some work or regenerate files when this changes can add this submodule file as a dependency.

efrotools.pyver.get_project_python_executable(projroot: Path | str) → str

Return the path to a standalone Python interpreter for this project.

In some cases, using sys.executable will return an executable such as a game binary that contains an embedded Python but is not actually a standard interpreter. Tool functionality can use this instead when an interpreter is needed.

efrotools.sync module

Functionality for syncing specific directories between different projects.

This can be preferable vs using shared git subrepos for certain use cases.

class efrotools.sync.Mode(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)

Bases: Enum

Modes for sync operations.

CHECK = 'check'

FORCE = 'force'

FULL = 'full'

LIST = 'list'

PULL = 'pull'

class efrotools.sync.SyncItem(src_project_id: str, src_path: str, dst_path: str | None = None)

Bases: object

Defines a file or directory to be synced from another project.

dst_path: str | None = None

src_path: str

src_project_id: str

efrotools.sync.add_marker(src_proj: str, srcdata: str) → str

Given the contents of a file, adds a ‘synced from’ notice and hash.

efrotools.sync.check_path(dst: Path) → int

Verify files under dst have not changed from their last sync.

efrotools.sync.get_dst_file_info(dstfile: Path) → tuple[str, str, str]

Given a path, returns embedded marker hash and its actual hash.

efrotools.sync.run_standard_syncs(projectroot: Path, mode: Mode, syncitems: Sequence[SyncItem]) → None

Run a standard set of syncs.

Syncitems should be a list of tuples consisting of a src project name, a src subpath, and optionally a dst subpath (src will be used by default).

efrotools.sync.string_hash(data: str) → str

Given a string, return a hash.

efrotools.sync.sync_paths(src_proj: str, src: Path, dst: Path, mode: Mode) → int

Sync src and dst paths.

efrotools.toolconfig module

Functionality for wrangling tool config files.

This lets us auto-populate values such as python-paths or versions into tool config files automatically instead of having to update everything everywhere manually. It also provides a centralized location for some tool defaults across all my projects.

efrotools.toolconfig.install_tool_config(projroot: Path, src: Path, dst: Path) → None

Install a config.

efrotools.util module

Misc util calls/etc.

Ideally the stuff in here should migrate to more descriptive module names.

efrotools.util.explicit_bool(value: bool) → bool

Simply return input value; can avoid unreachable-code type warnings.

efrotools.util.get_files_hash(filenames: Sequence[str | Path], extrahash: str = '', int_only: bool = False, hashtype: Literal['md5', 'sha256'] = 'md5') → str

Return a hash for the given files.

efrotools.util.get_string_hash(value: str, int_only: bool = False, hashtype: Literal['md5', 'sha256'] = 'md5') → str

Return a hash for the given files.

efrotools.util.is_wsl_windows_build_path(path: str) → bool

Return whether a path is used for wsl windows builds.

Building Windows Visual Studio builds through WSL is currently only supported in specific locations; namely anywhere under /mnt/c/. This is enforced because building on the Linux filesystem errors due to case-sensitivity issues, and also because a number of workarounds need to be employed to deal with filesystem/permission quirks, so we want to keep things as consistent as possible.

Note that said quirk workarounds WILL be applied if this returns true, so this check should be as specific as possible.

efrotools.util.readfile(path: str | Path) → str

Read a utf-8 text file into a string.

efrotools.util.replace_exact(opstr: str, old: str, new: str, count: int = 1, label: str | None = None) → str

Replace text ensuring that exactly x occurrences are replaced.

Useful when filtering data in some predefined way to ensure the original has not changed.

efrotools.util.replace_section(text: str, begin_marker: str, end_marker: str, replace_text: str = '', *, keep_markers: bool = False, error_if_missing: bool = True) → str

Replace all text between two marker strings (including the markers).

efrotools.util.writefile(path: str | Path, txt: str) → None

Write a string to a utf-8 text file.

efrotools.util.wsl_windows_build_path_description() → str

Describe where wsl windows builds need to live.

efrotools.xcodebuild module

Functionality related to Xcode on Apple platforms.

class efrotools.xcodebuild.SigningConfig(certfile: str, certpass: str)

Bases: object

Info about signing.

certfile: str

certpass: str

class efrotools.xcodebuild.XCodeBuild(projroot: str, args: list[str])

Bases: object

xcodebuild wrapper with extra bells and whistles.

run() → None

Do the thing.

efrotools.xcodebuild.project_build_path(projroot: str, project_path: str, scheme: str, configuration: str, executable: bool = True) → str

Get build paths for an xcode project (cached for efficiency).

Module contents

Build/tool functionality shared between all efro projects.

This stuff can be a bit more sloppy/loosey-goosey since it is not used in live client or server code.