bacommon.langstr package

Language-agnostic complex strings – the LangStrSpec authoring model.

“LangStr” is the name for this whole generation of string handling (formerly working-titled Lstr2); the legacy client translation class keeps the Lstr name, which stays reserved for it to avoid ambiguity. Within that generation the type split is semantic (see strings-asset-migration.md D28 in ballistica-internal):

• LangStrSpec (here) is the authoring spec form – a claim about a string carrying no guarantee that its asset-package is locally present (or even still exists). This is the currency for authoring surfaces and wire/model dataclasses; consuming ends verify/resolve before display.

• The native client babase.LangStr (a later optimized C++ port of this proven model) represents a verified-local string – holding one implies it is displayable there. Its .spec property projects back to this form (always valid); there is deliberately no public unverified->verified conversion.

The model lets us pass around minimal, language-independent representations of a complex string (substitutions, plurals, nesting) and resolve to a flat string in a particular language only at display – so one representation serves clients of any language.

type bacommon.langstr.EncodedLangStr = list[str | int | EncodedLangStr]

class bacommon.langstr.LangStrDir(apverid: str, tree: WrapperTree, prefix: str = '')

Bases: object

Runtime root/subdir accessor for a generated wrapper package.

exception bacommon.langstr.LangStrError

Bases: Exception

A malformed language-string or encode-context operation.

class bacommon.langstr.LangStrSpec

Bases: IOMultiType[LangStrSpecTypeID]

A deferred, language-agnostic complex string.

The base of a small multitype: a language-string is a LangStrSpecResource (an asset-package string addressed by apverid + logical name – the common authored form), a LangStrSpecValue (a raw literal that needs no package), or a LangStrSpecResourceIndexed (the compact integer-addressed projection of a resource, for contexts that carry a package-index map). All forms take keyword substitutions whose values may themselves be language-strings, so a LangStrSpec is a recursive tree; it holds tokens, not text, and only decodes to a flat string in some particular locale at display time.

Wire notes: the indexed form is the multitype default, so it alone serializes without a type tag (it is the space-sensitive form). Clients older than LANGSTR_EXT_MIN_BUILD understand only tag-free resource values with flat subs; producers that know the client build must gate everything beyond that (nested subs and the value/indexed forms) on it.

classmethod get_default_type_id() → LangStrSpecTypeID | None

Return a type-id to be assumed when none is present.

By default, dataclassio errors when deserializing multitype data that contains no type-id value. Overriding this to return a type-id changes that behavior: data with no type-id present will be deserialized as the returned type, and instances of that type will be serialized without a type-id value. This both saves a bit of space and allows ‘upgrading’ an existing regular dataclass to a multitype - simply designate the original dataclass type as the default and old serialized data will remain loadable (and data for the default type will remain loadable by old code).

Be aware of the following, however:

• Once serialized data exists anywhere without type-id values, the default type-id must never be changed or removed; doing so would cause that existing data to be silently reinterpreted as some other type (or to error).

• A missing type-id normally acts as a sanity check when deserializing; defining a default effectively disables that check, meaning malformed data may deserialize successfully as the default type instead of erroring.

classmethod get_type(type_id: LangStrSpecTypeID) → type[LangStrSpec]

Return the subclass for each of our type-ids.

classmethod get_type_id() → LangStrSpecTypeID

Return the type-id for this subclass.

classmethod get_type_id_storage_name() → str

Return the key used to store type id in serialized data.

The default is a short obscure value so that it is unlikely to conflict with members of individual type attrs, but in some cases one might prefer to serialize it to something simpler like ‘type’ by overriding this call. One just needs to make sure that no encompassed types serialize anything to that same name themself (dataclassio will error if they do).

class bacommon.langstr.LangStrSpecResource(apverid: str, name: str, subs: dict[str, str | int | ~bacommon.langstr._core.LangStrSpec]=<factory>)

Bases: LangStrSpec

An asset-package string: the common authored LangStrSpec form.

subs maps each substitution keyword to its value – a flat str / int or a nested LangStrSpec. A no-arg string has empty subs. The value carries its own exact apverid (so an encode context can discover the package union from the values themselves) and the string’s logical name (mapped to its integer index at encode time).

apverid: str

classmethod get_type_id() → LangStrSpecTypeID

Return the type-id for this subclass.

name: str

subs: dict[str, str | int | LangStrSpec]

class bacommon.langstr.LangStrSpecResourceIndexed(pkg: int, index: int, subs: list[str | int | LangStrSpec] = <factory>)

Bases: LangStrSpec

The compact integer-addressed projection of a resource string.

Usable only against a context carrying the {pkg_int: apverid} package-index map (and package structures for positional-sub ordering); see LanguageStringDecodeContext. Substitutions are positional here (canonical param order), matching the EncodedLangStr chunk model. This form is the multitype default, so it serializes without a type tag.

classmethod get_type_id() → LangStrSpecTypeID

Return the type-id for this subclass.

index: int

pkg: int

subs: list[str | int | LangStrSpec]

class bacommon.langstr.LangStrSpecTypeID(*values)

Bases: Enum

Type IDs for the LangStrSpec multitype’s forms.

RESOURCE = 'r'

RESOURCE_INDEXED = 'i'

VALUE = 'v'

class bacommon.langstr.LangStrSpecValue(value: str, subs: dict[str, str | int | ~bacommon.langstr._core.LangStrSpec]=<factory>)

Bases: LangStrSpec

A raw literal string value needing no asset package.

For server-generated dynamic text (player names, pre-formatted numbers, etc.) that rides a LangStrSpec-shaped slot without a package entry. The value is locale-independent; subs are substituted into {name} tokens exactly like a plain resource value (nested language-strings allowed).

classmethod get_type_id() → LangStrSpecTypeID

Return the type-id for this subclass.

subs: dict[str, str | int | LangStrSpec]

value: str

class bacommon.langstr.LanguageStringDecodeContext(package_index_map: dict[int, str], structures: dict[str, PackageStructure], language: dict[str, dict[str, str | StringSelector]], locale: Locale)

Bases: object

Decodes chunks into flat strings for one target locale.

Holds the {pkg_int: apverid} map (from the encoder), the package structures, and the per-apverid string values for a single locale. decode() resolves a chunk (recursively rendering nested values) via bacommon.loctext.evaluate().

decode(encoded: EncodedLangStr) → str

Resolve a chunk to a flat string in this context’s locale.

Fail-visible: any structural problem yields an LANGSTR_ERROR:… sentinel (and a logged warning) rather than crashing the caller.

decode_value(lstr: LangStrSpec) → str

Resolve any language-string form to flat text in this locale.

The tolerant all-forms counterpart of decode(): handles LangStrSpecResourceIndexed (via this context’s package-index map + structures), LangStrSpecValue (self-contained), and plain LangStrSpecResource (by name, for legacy/mixed payloads). Fail-visible like everything else on the decode side.

to_resource(lstr: LangStrSpec, _depth: int = 0) → LangStrSpec

Convert integer-indexed nodes back to the resource form.

The inverse of LanguageStringEncodeContext.to_indexed, for consumers that ingest indexed wire values but hold some of them in the self-describing name form (e.g. deferred client effects that outlive their containing payload’s package-index map). Returns new objects (resource/value nodes are rebuilt with converted subs); raises LangStrError for indices unknown to this context.

class bacommon.langstr.LanguageStringEncodeContext(lstrs: list[LangStrSpec], structures: dict[str, PackageStructure])

Bases: object

Encodes LangStrSpec values into minimal language-free chunks.

Built from the batch of values to send: it computes the union of apverids they reference (recursively – nested values know their own apverid) and assigns each a stable integer index. encode() then emits [pkg_int, str_int, …subs]; package_index_map is the only mapping the consumer needs (string indices resolve from the content-pinned apverid itself).

encode(lstr: LangStrSpec) → EncodedLangStr

Encode one value (recursively) into a minimal chunk.

property package_index_map: dict[int, str]

The {pkg_int: apverid} map a decoder needs.

to_indexed(lstr: LangStrSpec) → LangStrSpec

Convert a resource/value tree to its integer-indexed form.

Resource nodes become LangStrSpecResourceIndexed (with positional subs in canonical param order); literal LangStrSpecValue nodes pass through (with their nested subs converted). New objects are returned; the input tree is never mutated. Raises LangStrError loudly for packages/strings unknown to this context (authoring-side errors) or already-indexed input.

class bacommon.langstr.LanguageStringNameDecodeContext(language: dict[str, dict[str, str | StringSelector]], locale: Locale, *, param_kinds: dict[str, dict[str, dict[str, str]]] | None = None, components: dict[str, dict[str, str | StringSelector]] | None = None)

Bases: object

Decodes LangStrSpec values directly, by name, for one locale.

The name-based counterpart to LanguageStringDecodeContext: it resolves an in-memory LangStrSpec (carrying its apverid, string name, and keyword subs) straight against per-apverid per-locale values – no integer indices, package-index-map, or PackageStructure needed, since the subs are self-describing keyword->value pairs. This is the client’s primary path: resolve the referenced packages, gather their per-locale values, then decode each LangStrSpec in the client’s locale.

Fail-visible like LanguageStringDecodeContext – any structural problem yields an LANGSTR_ERROR:… sentinel (and a logged warning) rather than crashing the caller.

decode(lstr: LangStrSpec) → str

Resolve a LangStrSpec to a flat string in this locale.

Fail-visible: any structural problem yields an LANGSTR_ERROR:… sentinel (and a logged warning) rather than crashing the caller.

class bacommon.langstr.PackageDef(apverid: str, strings: tuple[StringDef, ...])

Bases: object

Language-free definition of one asset-package-version’s strings.

The shared source the encode/decode PackageStructure and the type-safe wrapper codegen both derive from (in the real system, from an apverid’s resolved listing; in tests, hand-built).

apverid: str

strings: tuple[StringDef, ...]

class bacommon.langstr.PackageStructure(apverid: str, strings: dict[str, tuple[str, ...]])

Bases: object

Language-free structure of one asset-package-version.

Maps string names <-> integer indices (assigned in canonical sorted-name order so both ends agree without shipping the mapping) and holds each string’s ordered substitution-keyword list. Carries no translations – encoding needs only this.

apverid

strings maps each logical name to its substitution keywords (() for a no-arg string). Order of the passed keywords is ignored: positional-substitution order is canonically alphabetical, enforced here so producer- and consumer-derived structures can’t disagree on it.

classmethod from_def(pkgdef: PackageDef) → PackageStructure

Build the encode/decode structure from a package definition.

classmethod from_language_values(apverid: str, values: dict[str, str | StringSelector]) → PackageStructure

Derive the structure from one locale’s complete value set.

The consumer-side counterpart of from_def(): string indices come from the canonical sorted-name order (the key set is identical across locales by construction – see complete_locale_values) and each string’s substitution keywords from its own value via bacommon.loctext.substitution_names(). Both ends canonicalize param order alphabetically, so a structure derived here agrees with the producer’s brief-derived one; producer tests lock that agreement.

index_of(name: str) → int

Return the integer index for a string name.

name_of(index: int) → str

Return the string name for an integer index.

params_of(name: str) → tuple[str, ...]

Return the ordered substitution keywords for a string name.

class bacommon.langstr.StringDef(path: str, params: tuple[tuple[str, str], ...] = (), docs: str = '', english: str = '')

Bases: object

One string’s language-free definition.

params is the ordered list of (keyword, kind) where kind is 'text' (a text sub -> str | LangStrSpec) or 'count' (the plural pivot -> int); () for a no-arg string. The canonical ordering (sorted keyword) is what fixes the positional substitution order.

docs (author usage docs) and english (an English preview of the rendered text) are optional docstring material for wrapper codegen only – neither participates in the encode/decode structure.

docs: str = ''

english: str = ''

params: tuple[tuple[str, str], ...] = ()

path: str

class bacommon.langstr.WrapParams(min_lines: int = 1, max_lines: int | None = None, max_chars_per_line: int | None = None)

Bases: object

Constraints for splitting a text value into lines client-side.

Mirrors the engine’s simple equal-width line splitter (babase.split_text_into_lines()): text is broken only at valid line-break opportunities, using the fewest lines that keep every line within max_chars_per_line (when provided) while staying between min_lines and max_lines (None means unlimited), with line lengths balanced within that count. So max_chars_per_line alone gives basic wrapping and min_lines alone gives an exact line count. Constraints are best-effort.

Default to pinning an exact line count: set min_lines to the layout’s designed count and leave max_chars_per_line unset. A max_chars_per_line-driven wrap yields a per-locale varying line count (translation lengths differ), which reads as broken in layouts designed around a specific count — and every legacy-converted string is such a layout, since the legacy pipeline hand-baked newlines at fixed counts (see D21 in the strings-asset-migration initiative). Reserve max_chars_per_line for surfaces explicitly designed to tolerate a variable number of lines.

Per decision D-t these are definition-time presentation hints: a string definition carries them optionally, they ride each locale blob, and evaluation applies them automatically. They are locale-invariant, and width-driven layout consumers may ignore them (they are a fallback presentation default).

max_chars_per_line: int | None = None

max_lines: int | None = None

min_lines: int = 1

type bacommon.langstr.WrapperTree = dict[str, tuple[str, ...] | WrapperTree]

bacommon.langstr.collect_apverids(lstr: LangStrSpec, acc: set[str]) → None

Gather every asset-package-version a language-string tree references into acc.

Indexed nodes resolve against an out-of-band context so they contribute no apverids themselves, but their substitution values are still walked (a resource-form node can appear anywhere in a mixed tree).

Note to implementers: keep this a module-level function; a self-recursive closure would create a reference cycle (function -> closure cell -> function) at every call site, adding cyclic-gc pressure the engine works hard to avoid.

bacommon.langstr.contains_resource_form(lstr: LangStrSpec) → bool

Return whether a language-string tree contains any full resource-form (non-indexed) node.

Used by consumers verifying that a wire payload claiming the integer-indexed form really is fully indexed (a resource-form leak means some producer path skipped indexing).

bacommon.langstr.convert_time_subs(subs: dict[str, 'str | int | LangStrSpec | datetime.datetime | datetime.timedelta'], now: datetime.datetime | None = None) → dict[str, 'str | int | LangStrSpec']

Convert any time-typed sub values to their wire form.

The integer-milliseconds wire value is an implementation detail of the duration machinery; conversion is driven purely by each value’s type (see time_sub_millis()), so no per-param kind knowledge is needed – the typed stubs are what hold authors to passing time types only for duration params. now is resolved at most once per call.

bacommon.langstr.data_size_str(bytecount: int, locale: Locale, values: dict[str, str | StringSelector], *, compact: bool = False) → str

Render a byte count as human-readable size in locale.

values is the components package’s per-locale value map. The ladder and the adaptive decimals (one place below ten units, none above) mirror efro.util.data_size_str(), as does compact for width-constrained slots: the only rung it changes is bytes, which renders through the abbreviated bytes_compact entry (“37 B”) instead of pluralizing (“37 bytes”) – larger rungs are already abbreviated. Abbreviations don’t inflect, so that entry is a plain {amount} template like the larger rungs, not a plural.

Raises KeyError if the components package is missing an entry – the decode path turns that into the usual fail-visible LANGSTR_ERROR sentinel rather than letting it escape.

bacommon.langstr.duration_str(millis: int, locale: Locale, values: dict[str, str | StringSelector], *, maxparts: int = 2, decimals: int = 0, direction: str | None = None, clamp: bool = False) → str

Render a length of time as composed abbreviated units.

Mirrors efro.util.timedelta_str (“1h 23m”; maxparts caps the composed units largest-first, decimals applies fractional places to the last one), rendering each part through the components package’s per-locale duration/ unit templates and joining with the curated duration_separator.

millis follows the signed target - now convention (D12), in integer milliseconds – the sub wire type must stay int, and ms precision is what keeps decimals meaningful. direction ('past' / 'future') renders the magnitude of that sign and floors the other to 0s – countdown-safe by construction; clamp floors negatives for an undirected length; with neither, negatives render as magnitude plus a leading - (D-neg, same caveats).

Raises KeyError if the components package is missing an entry – the decode path turns that into the usual fail-visible LANGSTR_ERROR sentinel rather than letting it escape.

bacommon.langstr.format_number(value: float, decimals: int, locale: Locale) → str

Render a number with fixed decimals and the locale’s mark.

Rounding happens first and the mark is swapped last, on a known ASCII rendering – never format-then-reparse, which would have to guess which of ./, it was looking at.

bacommon.langstr.package_structure(apverid: str, tree: WrapperTree) → PackageStructure

Build a PackageStructure from a wrapper’s runtime _TREE.

Flattens the nested tree into the {logical-path: param-keywords} map the encode/decode contexts need – so a consumer of a vendored package just passes module.APVERID, module._TREE (both module-level).

bacommon.langstr.parse_language_blob(text: str) → dict[str, str | StringSelector]

Parse a canonical language blob into a {name: value} map.

The exact inverse of serialize_language_blob(): reads the top-level strings object, turning each value back into a str (plain) or a StringSelector (a dict). A blob with no strings key (e.g. a legacy-only package) yields an empty map; malformed values are skipped (fail-soft on the consumer side).

bacommon.langstr.parse_language_components(text: str) → dict[str, str | StringSelector]

Read the build-embedded formatter components out of a blob.

Same value shapes as parse_language_blob() (plain str or StringSelector), read from the sibling components key. Absent on any package with no spec’d params, and on every blob written before components existed – both yield an empty map rather than an error.

bacommon.langstr.parse_language_param_kinds(text: str) → dict[str, dict[str, str]]

Read the {name: {param: kind}} map out of a language blob.

The display-side counterpart of param_kinds in serialize_language_blob(). Only strings carrying a non-text param appear; everything else is absent and reads as plain text substitution, which is what a blob written before this existed yields for every entry. Fail-soft throughout, matching parse_language_blob() – a malformed entry is skipped rather than failing the whole blob.

bacommon.langstr.render_display_param(kindexpr: str, value: str | int | float, locale: Locale, components: dict[str, str | StringSelector]) → str

Render one spec’d param value for a locale – the shared dispatch.

kindexpr is the display-kind expression a blob’s k carrier (or a wrapper’s baked kinds map) holds – the bare kind or kind plus spec args ('bytes(compact=true)'); see display_kind. The single render-dispatch both the server-side decode context and the client wrapper runtime route through, so the two sides can’t drift.

Raises on a malformed expression, an unknown kind, or a missing component entry; callers apply their own fail-visible or fail-soft policy.

bacommon.langstr.serialize_language_blob(values: dict[str, str | StringSelector], wraps: dict[str, WrapParams] | None = None, param_kinds: dict[str, dict[str, str]] | None = None, components: dict[str, str | StringSelector] | None = None) → str

Serialize a per-locale value map to the canonical language blob.

values maps each string’s logical name to its value – a plain str or a StringSelector. wraps optionally maps names to their definition-time WrapParams (decision D-t); a wrapped entry is emitted as a {'v': value, 'w': wrap} carrier dict (which pre-wrap clients skip fail-soft). Output is deterministic (sorted keys, fixed formatting) for cache stability and diffability.

param_kinds optionally maps a name to its {param: kind} for params whose kind the display side must know – a byte count rendered as “1.2 GB” cannot be recovered from the translated text, which carries only a {name} token. It rides the same carrier under 'k'.

Pass only the params that actually need it (anything but 'text'). A string with none is emitted exactly as before, so adding this leaves every existing blob byte-identical – which matters because blob content is a cache key.

bacommon.langstr.time_sub_millis(value: datetime | timedelta, now: datetime | None = None) → int

One time-typed sub value’s wire form (signed ms int).

The single shared arithmetic for wrapper accessors’ useful duration types: a datetime.timedelta is already a signed length; a datetime.datetime is an absolute time, converted to signed target - now per D12 so the string’s dir handles past/future. now defaults to efro.util.utc_now(); batch callers pass one shared value so a page of renders can’t drift against itself. Datetimes must be timezone-aware (naive ones raise from the subtraction, per stdlib rules).