Python Runtime, Imports, and Systems Notes

Python invocation decides both the code that executes and the first entry on sys.path. The surrounding notes cover supporting runtime concepts encountered when building tools and static sites.

Choose a Python Invocation Form #

Python builds sys.path from a context-dependent first entry, PYTHONPATH when set, the standard library and extension locations, and site-specific paths. Options such as -S and -P alter that construction.

-P or PYTHONSAFEPATH prevents Python from prepending the script directory or current directory. Use it in commands where importing an adjacent, untrusted file would be unsafe.

Run an application as an installed module rather than executing an internal file:

project/
  pyproject.toml
  src/
    example/
      __init__.py
      __main__.py
      cli.py

python -m example

Module execution preserves package context, so explicit relative imports work. Executing src/example/cli.py directly sets __package__ differently and breaks imports such as from . import config.

Resolve Imports Predictably #

For import a.b.c, Python resolves top-level a from sys.path, then looks for b in a, and finally c in b. The resulting module object is cached in sys.modules; later imports of the same fully qualified name reuse it.

An absolute import starts at sys.path:

from example import cli

A relative import starts at the current package:

from . import cli
from ..shared import config

Avoid local names such as json.py, email.py, or requests.py. Such files can shadow the standard library or an installed dependency depending on the invocation directory. Diagnose resolution with:

python -c 'import sys; print(*sys.path, sep="\n")'
python -c 'import example; print(example.__file__)'

An __init__.py creates a regular package and can define a deliberately small public API. Namespace packages can span distribution boundaries without that file, but a regular package has clearer ownership for an application. __main__.py is the entry point for python -m package.

Package Applications and Dependencies #

Define a console script in pyproject.toml for a stable installed command:

[project.scripts]
example = "example.cli:main"

The generated launcher imports the project from its installation environment, so it does not depend on the caller’s current directory.

pip resolves all declared requirements into one compatible graph. A private package service can proxy PyPI, host internal projects, or provide both. Put credentials in a keyring, environment variable, user configuration, or CI secret rather than in pyproject.toml:

python -m pip install \
  --index-url 'https://token@packages.example.test/api/pypi/simple' \
  internal-package

Do not print that URL in build logs. For public and private dependencies, prefer one trusted virtual index that proxies PyPI over an untrusted --extra-index-url: pip considers candidates from configured indexes without giving an index a security priority. A public project that reuses an internal name can otherwise be selected.

Use constraints or a lock file to make builds repeatable:

python -m pip install -r requirements.txt -c constraints.txt
python -m pip check

pip check validates installed declared requirements. A clean virtual environment exposes missing declarations and accidental global imports.

Select a Regular-Expression Dialect #

Regular expressions have incompatible dialects. POSIX Basic Regular Expressions (BRE), POSIX Extended Regular Expressions (ERE), and Perl Compatible Regular Expressions (PCRE) share concepts but not syntax or features.

POSIX matching chooses the leftmost longest match. A PCRE-style engine often uses ordered backtracking and honors lazy quantifiers. GNU sed defaults to BRE, sed -E selects ERE, and grep -P requests a PCRE-compatible engine when that build supports it. grep -P remains line-oriented unless input is deliberately transformed.

Choose the dialect exposed by the tool rather than copying a pattern from a different engine. Quote shell arguments so the shell cannot interpret backslashes, brackets, dollar signs, or exclamation marks before the regex engine sees them.

Avoid Races in Bit Interleaving #

Consider a zeroed byte array where one thread sets all even-numbered bits and another sets all odd-numbered bits. A non-atomic read-modify-write sequence is not safe:

old = byte
byte = old | mask

Both threads can read zero. One writes an even-bit mask and the other writes an odd-bit mask, so the later store overwrites the earlier store. The final byte can lack bits even though the masks do not overlap. In C or C++, a data race has undefined behavior; POSIX threads do not make ordinary memory updates atomic.

Use an atomic read-modify-write operation:

atomic_fetch_or_explicit(&words[index], mask, memory_order_relaxed);

memory_order_relaxed is enough when the only requirement is an atomic accumulation. Use acquire-release synchronization only when the flag also publishes or consumes related data.

Atomicity does not remove cache contention. Adjacent words can share a cache line, and writes from separate cores then cause coherence traffic known as false sharing. Partition the array by cache-line-sized regions, batch masks per word, or assign one owner per word when throughput matters.

Drive GLib’s Main-Loop Model #

GLib separates event sources, contexts, and loops:

• A GSource represents an event producer such as an idle callback, timeout, file descriptor, or child-process watch.
• A GMainContext owns sources and performs prepare, poll, check, and dispatch iterations.
• A GMainLoop repeatedly iterates a context until asked to quit.

The thread-default context matters for asynchronous APIs. Push the intended context before creating an operation when callbacks must return to a particular thread:

g_main_context_push_thread_default(context);
/* Create asynchronous operations here. */
g_main_context_pop_thread_default(context);

Attach a source to the selected context and retain ownership rules carefully:

GSource *source = g_timeout_source_new_seconds(1);
g_source_set_callback(source, on_tick, user_data, NULL);
g_source_attach(source, context);
g_source_unref(source);

Calling g_main_loop_run() on a context that another thread owns can deadlock or violate thread-affinity assumptions. Move work between threads with an explicit synchronization design, then invoke a callback in the destination context.

Define GObject Types and Signals #

GObject combines a runtime type registry, inheritance, interfaces, properties, and signals. A final type has an instance structure and class structure that are not intended for subclasses:

G_DECLARE_FINAL_TYPE(FooBar, foo_bar, FOO, BAR, GObject)

The declaration provides foo_bar_get_type(), cast and type-check macros, and autoptr cleanup support. G_DEFINE_FINAL_TYPE or G_DEFINE_TYPE in one C translation unit implements registration and requires foo_bar_init() and foo_bar_class_init().

For a derivable public base type, use G_DECLARE_DERIVABLE_TYPE. The public instance structure must contain only the parent instance because subclasses embed it. Store implementation fields in private data:

G_DEFINE_TYPE_WITH_PRIVATE(FooBar, foo_bar, G_TYPE_OBJECT)

G_ADD_PRIVATE registers the private allocation, and the generated foo_bar_get_instance_private() retrieves it. An abstract derivable type uses G_DEFINE_ABSTRACT_TYPE.

An interface declaration uses G_DECLARE_INTERFACE; its implementation uses G_DEFINE_INTERFACE and supplies a default-interface initializer. A concrete type adds an interface with G_IMPLEMENT_INTERFACE inside G_DEFINE_TYPE_WITH_CODE. That macro emits registration code during *_get_type_once(), so registration happens before callers instantiate the type.

Register a signal in class initialization with g_signal_new, connect a handler with g_signal_connect, and emit it with g_signal_emit or a generated signal function. Validate signal names, argument types, and the thread that emits the signal. GActionMap owns named actions, while GActionGroup exposes actions for activation and state changes; applications often implement both to register actions locally and publish them to UI code.

Build Hugo Sites from Templates #

Hugo parses content and configuration, builds a page collection, resolves templates, renders output formats, and writes files to the publish directory. Page rendering can depend on site-wide collections such as taxonomies and regular pages, so treat a build as a site computation rather than a sequence of isolated Markdown conversions.

Template lookup uses kind, output format, language, and layout naming. A layout in layouts/ overrides a theme layout of the same lookup specificity. Use hugo --templateMetrics --templateMetricsHints to locate expensive templates and hugo --renderToMemory when a process must inspect generated content without publishing it.

Go template variables, functions, and pipelines compose data:

{{ $pages := where site.RegularPages "Section" "posts" }}
{{ range $pages.ByTitle }}
  <a href="{{ .RelPermalink }}">{{ .LinkTitle }}</a>
{{ end }}

with changes the dot context only when its value is nonempty. Save the outer context in $ or a named variable before entering nested ranges. Escape rules depend on template type: HTML templates contextually escape output, while using safeHTML or safeURL asserts that an input has already been validated. Do not mark author-controlled or remote data safe merely to make markup render.

Shortcodes are content-level templates. Angle-bracket calls render their result as markup, while percent-delimited calls can cause the shortcode output to participate in Markdown rendering. Use a normal Markdown section, list, table, or blockquote when the content has no reusable rendering behavior. Avoid copying a shortcode’s CSS or JavaScript into content; keep behavior in the site implementation and keep article prose portable.

Encode Transitions on a Physical Link #

Line coding converts bits into a physical signal with timing properties. NRZ encodings avoid returning to a baseline between bit cells, which saves bandwidth but can lose clock recovery during long runs without transitions.

In NRZI, a transition represents one bit value and no transition represents the other. The convention varies by protocol, so state the mapping instead of assuming it:

Current level changes at a bit boundary  -> encoded one
Current level remains unchanged          -> encoded zero

USB uses a related convention with bit stuffing: after six consecutive ones, the transmitter inserts a zero to force a transition. Optical-disc channels also use run-length-limited coding and error correction because a physical medium cannot reliably represent arbitrary transition spacing.

Read a Large C Project by Build Boundaries #

Generated sources, portability layers, and plugin entry points explain more of a C project than its directory names. Sudo provides useful examples:

• include/sudo_queue.h is a compatibility copy of BSD queue macros for singly linked, tail, doubly linked, and doubly linked tail queues.
• lib/utils/ contains prefixed implementations and fallback functions. A compatibility header can map an unavailable getdelim to sudo_getdelim, so callers retain a portable API.
• Parser files such as gram.c and gram.h derive from gram.y; lexer files derive from Flex input. Build flags can set a parser prefix, explaining a generated symbol name absent from handwritten source.
• Release tarballs retain generated files so a normal build does not require Flex, Bison, or a development-only regeneration mode.

Read Makefile.in and generated-file comments before modifying a generated C file. Change the grammar, lexer, or input definition, regenerate through the project’s documented build target, and review the generated diff.

Curate Changelogs as Durable Records #

A changelog should describe a released behavior, compatibility effect, or operator action. Development diary entries, abandoned experiments, and unverified links do not provide release value.

For native static linking, distinguish archive production from executable linking. Autoconf options such as --disable-shared --enable-static control whether a project builds shared libraries and archives; they do not guarantee that a final executable has no dynamic dependencies. Inspect the actual link command with make -n or the build’s verbose mode, then pass linker flags where the final executable link is performed. Verify the result with the platform’s dependency inspection tool, such as ldd on Linux or otool -L on macOS. Static linkage can still fail when a required static archive or system component is unavailable.