Augmentation

augmentation

Array-level augmentation for ML pipelines.

Operates on pre-extracted NumPy arrays without Bvh objects. All functions accept any rotation representation supported by pybvh: "quat", "6d", "axisangle", "rotmat", or "euler". Euler arrays additionally require an euler_orders kwarg.

Every function takes a :class:~pybvh_ml.MotionArrays as its single positional argument and returns a new one; every other parameter is keyword-only. The container is a distinct type, so passing it positionally cannot be confused with anything else — whereas the loose (root_pos, joint_data) pair it replaces was two shape-compatible ndarrays a swapped call would have silently corrupted. Call with rotate_vertical(arrays, angle=..., up_axis=..., representation=...).

Angles are in radians by default, matching pybvh's convention. The functions that take one accept degrees=True to interpret it in degrees instead, mirroring pybvh's own opt-in flag.

Streams. A sample may carry rotations, positions, or both, and the governing rule is that a step must handle every stream the sample carries, or refuse it — a pipeline never carries a stream a step left behind. Each function declares what it handles with :func:handles_streams, and the declaration is checked before the step runs. All four geometric steps (:func:rotate_vertical, :func:mirror, :func:speed_perturbation_arrays, :func:dropout_arrays) and both noise functions handle every stream; only the two keypoint-jitter functions decline anything. representation is consequently str | None throughout, required only when the sample carries joint_rot.

Output dtype. Every function here computes in float64 — pybvh's dtype, and the only one its conversions are exact in — and returns each stream in the dtype it was given, per stream, so a float32 clip comes back float32 without the arithmetic having been done in single precision. :class:~pybvh_ml.AugmentationPipeline does the same across a whole run, which is what keeps a result's dtype independent of which probabilistic steps fired for that sample. Widening is lossless, so the float32 result is exactly the float64 result narrowed. The per-stream rule earns its keep with positions: float32 keypoints beside float64 rotations is the ordinary ST-GCN case.

Output storage. A function's result may share storage with its input for a stream it does not touch — :func:add_root_position_noise returns the input's joint_rot by reference under skeleton-centered positions, and the keypoint-jitter functions return the stream they were not pointed at. This is safe rather than sloppy: :class:~pybvh_ml.MotionArrays fields are read-only views, so no caller (this package included) can write through the shared buffer, and the alternative — copying every untouched stream — would allocate a full clip per step for nothing. The stronger guarantee, freshly allocated arrays regardless of what ran, belongs to :class:~pybvh_ml.AugmentationPipeline, which is the surface a data loader calls; reach for it (or np.array(...)) if you need storage you own.

handles_streams(*streams: str) -> Callable[[Callable], Callable]

Declare which :class:~pybvh_ml.MotionArrays streams a step handles.

The coherence rule this enforces: a step must handle every stream the sample carries, or refuse it. A pipeline never carries a stream a step left behind, so a rotation-only step meeting a sample with positions raises rather than returning positions that no longer match the rotations beside them.

"Handles" means the stream is left correct, by either of two routes:

• transformed from its own input — what all four geometric steps do (:func:rotate_vertical, :func:mirror, :func:speed_perturbation_arrays, :func:dropout_arrays); or
• re-derived from another stream — what :func:add_joint_rotation_noise does, replacing the position streams with forward kinematics of the noised rotations rather than transforming the incoming ones.

It does not mean the positions stay the exact forward kinematics of the rotations beside them, except immediately after a re-derivation. Two divergences are intrinsic to the math, and every pipeline in the field has them:

• Mirror. Positions reflect exactly in world space; rotations reflect in parent-local space. The two agree when the rest pose is laterally symmetric and diverge on rigs with asymmetric offsets, with the error accumulating down the chain. Each stream stays individually correct; the pair stops being FK partners.
• Speed perturbation and dropout. Positions are linearly interpolated, rotations slerped — chord versus arc. They agree at the knots and drift between them.

And a re-derivation discards whatever stream-specific history the positions carried: on a rig with asymmetric rest offsets, [mirror, add_joint_rotation_noise] ends with FK of locally-mirrored rotations, throwing away the world-exact reflection the position stream held, while [add_joint_rotation_noise, mirror] keeps it. Both are defensible and neither is a bug, but a user who does not know reads the difference as one.

Undeclared steps default to {"root_pos", "joint_rot"}. Decorate a custom step once it genuinely transforms the position streams too::

@handles_streams("root_pos", "joint_rot", "joint_pos")
def my_step(arrays, *, scale):
    return arrays.replace(joint_pos=arrays.joint_pos * scale)

Parameters:

Name	Type	Description	Default
*streams	str	Names from :data:~pybvh_ml.arrays.STREAM_NAMES. The packers' derived names ("joint_vel" and friends) are not valid here: they are computed after all augmentation has run, so no step can transform them. Declare the position stream they come from and the derivative follows.	()

Returns:

Type	Description
callable	The decorator, which records the declaration on the function and returns it unchanged.

See Also

stream_support : Read a step's declaration back.

stream_support(fn: Callable) -> frozenset[str]

The streams fn declares it handles.

{"root_pos", "joint_rot"} for a step that declares nothing — see :func:handles_streams for what "handles" means and why that is the right default.

Parameters:

Name	Type	Description	Default
fn	callable	An augmentation step.	required

Returns:

Type	Description
frozenset of str

rotate_vertical(arrays: MotionArrays, *, angle: float, up_axis: str, representation: str | None = None, degrees: bool = False, euler_orders: list[str] | None = None) -> MotionArrays

Rotate joint arrays around the vertical axis.

In rotation space only the root joint (index 0) and root position are modified; non-root joints are in parent-local space and stay unchanged. Position streams are all rotated, since they are world coordinates rather than parent-local ones — this is the one place where the rotation and position analogs differ structurally.

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	Any combination of streams. representation is required only when joint_rot is present.	required
angle	float	Rotation angle in radians, or degrees when degrees=True.	required
degrees	bool	Interpret angle in degrees. Default False (radians), the convention throughout pybvh and pybvh-ml; the flag exists so a caller whose configs are written in degrees can pass them straight through instead of converting at every call site.	False
up_axis	str	Signed axis string: '+x', '-x', '+y', '-y', '+z', or '-z'. The sign flips the rotation direction, so '+y' and '-y' produce opposite yaws for the same angle. Typically bvh.world_up.	required
representation	str	One of "quat", "6d", "axisangle", "rotmat", "euler". Required when arrays carries joint_rot; ignored for a positions-only sample.	None
euler_orders	list of str	Per-joint Euler order strings (e.g. ["ZYX", "ZYX", ...]). Required when representation="euler", ignored otherwise.	None

Returns:

Type	Description
MotionArrays

Notes

The rotation is about the world origin, not about the character: root_pos is rotated as a set of points, so a clip whose root sits away from the origin sweeps along an arc rather than turning on the spot. The alternative convention is a pivot at the character — typically the first frame's root projected to the ground plane — which is turn-in-place.

The two coincide exactly when the clip's first-frame root is at the origin, which is what center_root=True produces, so the packing and Dataset paths (where centering is the default) already get turn-in-place from this function. On uncentered arrays, center before rotating and add the offset back if that is what you want.

The same origin caveat applies to the position streams, and it is where a position_centering="skeleton" clip differs: those positions are root-relative, so the rotation is always about the character and always turn-in-place, whatever the root trajectory does. The function needs no knowledge of the centering to be correct — a rotation about the origin is linear, so it commutes with the constant shift that distinguishes the three frames.

mirror(arrays: MotionArrays, *, lr_joint_pairs: list[tuple[int, int]] | None = None, lr_node_pairs: list[tuple[int, int]] | None = None, lateral_axis: str, representation: str | None = None, euler_orders: list[str] | None = None) -> MotionArrays

Mirror joint arrays left-right.

Swaps left and right vertex data, negates the lateral component of root translation and of every position vertex, and reflects each rotation across the sagittal plane.

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	Any combination of streams. representation is required only when joint_rot is present.	required
lr_joint_pairs	list of (int, int)	[(left_idx, right_idx), ...] in joint-array space, typically :func:pybvh_ml.get_lr_pairs. The reflection is applied to every joint; this list only controls which ones additionally swap slots. That is the right behavior for midline joints (hips, spine, neck, head), which have no partner and must reflect in place. It is the wrong behavior for a lateral joint whose partner is missing from the list: it reflects in place instead of moving to its partner's slot, producing a pose where that limb mirrors while the rest of the body does not — silently, with no shape or value error. Completeness of this list is the caller's responsibility; an empty list reflects every joint in place without any swap, which is correct only for a skeleton that is entirely midline. Required when the sample carries a joint-space stream (joint_rot or joint_pos); None is refused there rather than treated as "no pairs", since the two produce different motion and only one of them is ever intended.	None
lr_node_pairs	list of (int, int)	The same list in node index space — joints and their end sites — typically skeleton_info["node_lr_pairs"] (from :attr:pybvh.Bvh.node_lr_pairs). Required when the sample carries node_pos, and not interchangeable with lr_joint_pairs: node indices diverge from joint indices as soon as any end site precedes a paired joint in file order, so the wrong list silently swaps the wrong vertices.	None
lateral_axis	str	Signed axis string: '+x', '-x', '+y', '-y', '+z', or '-z'. The sign is accepted for API symmetry with :func:rotate_vertical but does not affect the result (mirror is sign-invariant).	required
representation	str	One of "quat", "6d", "axisangle", "rotmat", "euler". Required when arrays carries joint_rot; ignored for a positions-only sample.	None
euler_orders	list of str	Required when representation="euler", ignored otherwise.	None

Returns:

Type	Description
MotionArrays

Notes

Mirroring is done in parent-local rotation space — reflect each rotation, swap the L/R slots — matching :func:pybvh.transforms.mirror, so mirroring arrays and mirroring the source :class:~pybvh.Bvh give the same motion. The alternative is mirroring in world space: run FK, reflect the resulting joint positions, and re-solve for local rotations. The two agree exactly when the rest pose is laterally symmetric (every left joint's offset is the mirror of its right partner's, and midline offsets have no lateral component), which holds for most retargeted rigs. They diverge on rigs with asymmetric offsets: the local-space result is still a valid pose, but not the exact reflection of the input, with the error accumulating down the chain from the first asymmetric bone. Neither pybvh nor pybvh-ml implements the world-space variant.

The position streams take the other route, because they have no choice: a position is a world coordinate, so reflecting it is the world-space mirror — negate the lateral component, swap the paired vertices. Each stream therefore stays individually correct while the pair stops being exact FK partners on an asymmetric rig. Recompute the positions from the mirrored rotations (or run :func:add_joint_rotation_noise, which re-derives them) if you need them to agree.

lateral_axis must also be given explicitly here, whereas :meth:pybvh.Bvh.mirror auto-detects it by averaging left-minus-right rest-pose offsets — again, an array-level function has no rest pose to measure. The default lateral_axis='+x' on :meth:~pybvh_ml.AugmentationPipeline.standard is a convention, not a measurement: it is correct for the usual Y-up / Z-forward rig and wrong for a rig whose lateral axis is Z. When in doubt, mirror one clip on the :class:~pybvh.Bvh (which measures the axis) and compare against the array path before trusting the assumed axis across a dataset.

add_joint_rotation_noise(arrays: MotionArrays, *, sigma: float, representation: str | None = None, degrees: bool = False, rng: np.random.Generator | None = None, euler_orders: list[str] | None = None, fk_topology: object | None = None, world_up: str | None = None) -> MotionArrays

Add Gaussian rotation noise to every joint.

For each joint at each frame, generates a small random rotation (axis uniformly random on the unit sphere, angle sampled from N(0, sigma)) and composes it with the original rotation: q_noisy = q_noise * q_original.

When the sample also carries positions, they are re-derived by forward kinematics from the noised rotations rather than transformed — the one step that handles a stream by re-derivation (see :func:handles_streams). That is what keeps the two streams FK partners, and it is only possible in this direction: rotation → position is FK, position → rotation would be IK.

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	Must carry joint_rot. root_pos passes through unchanged — jittering the root trajectory is :func:add_root_position_noise, a separate function because its sigma is a length rather than an angle.	required
sigma	float	Standard deviation of the rotation noise, in radians (or degrees when degrees=True).	required
representation	str	One of "quat", "6d", "axisangle", "rotmat", "euler". Required here, since this step cannot run on a sample without rotations at all.	None
degrees	bool	Interpret sigma in degrees. Default False (radians).	False
rng	numpy Generator	sigma=0 is a no-op in value but still draws from the generator, so a seeded pipeline's draw sequence does not depend on the sigma it was configured with. (:func:pybvh.transforms.add_rotation_noise short-circuits instead; ours cannot without changing the stream of every already-seeded pipeline.)	None
euler_orders	list of str	Required when representation="euler", ignored otherwise.	None
fk_topology	FkTopology	The skeleton as plain arrays, for the position refresh. Required when the sample carries a position stream and ignored otherwise. Rebuild it once per dataset from skeleton_info["fk_topology"] with :func:~pybvh_ml.build_fk_topology; :meth:~pybvh_ml.AugmentationPipeline.standard wires it for you.	None
world_up	str	Signed world-up axis (bvh.world_up, persisted as skeleton_info["world_up"]). Read only under position_centering="first", which needs to know which coordinate ground-plane centering leaves alone. pybvh 0.8.2 removed the '+y' default precisely because it silently mis-centered non-y-up skeletons, and an FkTopology carries no gravity direction, so there is nothing to fall back to.	None

Returns:

Type	Description
MotionArrays

Notes

Cost of the refresh. Roughly 0.9 ms per sample at F=64 on a 31-joint rig (0.64 ms FK plus 0.25 ms 6d→euler), ~1.7 ms at F=128 — about a tripling of per-sample augmentation cost when it fires. Dataloader workers absorb it, but it is not free, so it runs only when a position stream is actually present.

The noise model is isotropic in rotation space: one random axis per joint per frame, with the magnitude drawn from N(0, sigma). The alternative — and what :func:pybvh.transforms.add_rotation_noise does — is independent Gaussian noise on each Euler channel, which is cheaper but anisotropic (its effective magnitude depends on the channel order and on how near the pose is to gimbal lock), and is not even well defined for a 6d or quat array. This function is representation-agnostic, so it takes the isotropic route. The two agree in distribution only in the small-angle limit for a joint whose rotation is near identity.

See Also

add_root_position_noise : The root-translation counterpart. add_joint_position_noise : Keypoint jitter, for a sample with no rotations to noise.

add_root_position_noise(arrays: MotionArrays, *, sigma: float, rng: np.random.Generator | None = None) -> MotionArrays

Add Gaussian noise to the root translation.

Split from :func:add_joint_rotation_noise because the two sigmas are in different units — radians there, the data's length unit here — so a single degrees= flag could only ever have applied to one of them. pybvh made the same split for the same reason in 0.8.1 (add_noise → add_rotation_noise + add_position_noise).

To reproduce a combined call, chain them with the same generator, rotation first::

arrays = add_joint_rotation_noise(arrays, sigma=s, rng=rng,
                                  representation="6d")
arrays = add_root_position_noise(arrays, sigma=p, rng=rng)

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	joint_rot passes through untouched, and may be None. Position streams move with the root or not, depending on position_centering — see the Notes.	required
sigma	float	Standard deviation of the noise added to root_pos, in the data's positional units. There is no degrees= here because this is a length, not an angle.	required
rng	numpy Generator	sigma=0 is a no-op and draws nothing, matching :func:pybvh.transforms.add_position_noise and preserving what the fused add_joint_noise(sigma_pos=0) did.	None

Returns:

Type	Description
MotionArrays

Raises:

Type	Description
ValueError	If the sample carries positions and position_centering is None. This is one of the three surfaces whose correctness depends on the frame, and guessing is worse than refusing.

Notes

The step is centering-aware, which makes it exactly correct in every mode rather than merely allowed:

• "world" / "first" — a translation of the root translates every joint by the same amount, so the identical per-frame offset is added to every position vertex. One broadcast, exact.
• "skeleton" — positions are root-relative and genuinely do not move. Exact by construction.

Declining position streams outright would have been the conservative alternative, and it was rejected because it makes root noise unusable on skeleton-centered data, where it is trivially correct. The failure it prevents is quiet: under "world" centering, jittering root_pos while leaving the positions alone leaves the two streams mutually inconsistent — and under the canonical ST-GCN pack streams=("joint_pos",), where root_pos is not packed at all, it degrades further into an augmentation the model never sees.

See Also

add_joint_rotation_noise : The joint-rotation counterpart. add_joint_position_noise : Per-vertex keypoint jitter, which is a different augmentation — independent noise per joint rather than one offset shared by the whole body.

add_joint_position_noise(arrays: MotionArrays, *, sigma: float, rng: np.random.Generator | None = None) -> MotionArrays

Add independent Gaussian jitter to every joint position.

The keypoint jitter of the skeleton-action-recognition world: each vertex at each frame moves by its own draw, which is what models a pose estimator's per-joint error. Distinct from :func:add_root_position_noise, where one offset per frame moves the whole body rigidly.

Declines joint_rot-carrying samples, and the refusal is the governing asymmetry of this whole surface: positions are derived from rotations, so rotation → position is computable (forward kinematics) while position → rotation is not (inverse kinematics). A jittered position stream cannot be pushed back into the rotations beside it, so a sample carrying both is refused rather than left incoherent — jitter the rotations with :func:add_joint_rotation_noise instead, which re-derives the positions.

That refusal also closes the one composition that would be destructive: keypoint jitter can never be silently wiped by a later FK refresh, because the two steps can never share a pipeline.

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	Must carry joint_pos and must not carry joint_rot or node_pos.	required
sigma	float	Standard deviation of the per-vertex noise, in the data's positional units. 0 is a no-op and draws nothing, matching :func:add_root_position_noise.	required
rng	numpy Generator		None

Returns:

Type	Description
MotionArrays

Notes

position_centering is not read: independent per-vertex noise is correct in every frame, since the frames differ by a shift that commutes with adding noise.

See Also

add_node_position_noise : The node-space counterpart.

add_node_position_noise(arrays: MotionArrays, *, sigma: float, rng: np.random.Generator | None = None) -> MotionArrays

Add independent Gaussian jitter to every node position.

The node-space counterpart of :func:add_joint_position_noise — same math, applied to the stream that includes end sites (fingertips, toe tips, head top). See that function for why the two are separate names rather than one streams= kwarg, and for why both decline joint_rot.

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	Must carry node_pos and must not carry joint_rot or joint_pos.	required
sigma	float	Standard deviation of the per-vertex noise, in the data's positional units.	required
rng	numpy Generator		None

Returns:

Type	Description
MotionArrays

See Also

add_joint_position_noise : The joint-space counterpart.

speed_perturbation_arrays(arrays: MotionArrays, *, factor: float, representation: str | None = None, euler_orders: list[str] | None = None) -> MotionArrays

Speed perturbation via time resampling.

Uses SLERP for rotation interpolation (via quaternion space) and linear interpolation for root position and every position vertex.

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	Any combination of streams; every present one is resampled onto the same time stencil, so they cannot come out of step.	required
factor	float	Speed factor. > 1 = faster (fewer frames), < 1 = slower (more frames).	required
representation	str	One of "quat", "6d", "axisangle", "rotmat", "euler". Required when arrays carries joint_rot; ignored for a positions-only sample.	None
euler_orders	list of str	Required when representation="euler", ignored otherwise.	None

Returns:

Type	Description
MotionArrays	Arrays of F' frames. For F >= 2, F' = max(2, round(F / factor)) (Python banker's rounding — round(2.5) == 2). Inputs with fewer than 2 frames have nothing to interpolate between and are returned as unchanged copies.

Notes

Rotations are interpolated with pybvh.rotations.quat_slerp under its shortest=True default: the interpolant takes the short arc between adjacent frames, so a turn is never read as its

180° complement. The alternative, shortest=False, preserves a genuine wind-up or spin that exceeds half a turn; it is not exposed here because a per-frame stencil cannot tell one from the other.

A consequence for representation="quat" specifically: output quaternions may come back as -q relative to the corresponding input, since q and -q are the same rotation and the short arc picks whichever hemisphere is nearer. Every other representation is unaffected (the sign is not observable in them). Compare rotations, not raw components, when diffing output against input — or run the input through pybvh.rotations.quat_unwrap first, which makes a sequence hemisphere-continuous.

Positions are interpolated linearly, rotations slerped — chord versus arc. The two agree at the knots and drift between them, so resampled positions are not the exact forward kinematics of the resampled rotations beside them. This is intrinsic (there is no interpolant that is simultaneously the geodesic in rotation space and linear in position space), it is what every pipeline in the field does, and standardize_length(method="resample_linear") documents the same split from the other direction.

dropout_arrays(arrays: MotionArrays, *, drop_rate: float, representation: str | None = None, rng: np.random.Generator | None = None, euler_orders: list[str] | None = None) -> MotionArrays

Frame dropout with SLERP interpolation.

Randomly drops frames and fills the gaps with SLERP-interpolated rotations (via quaternion space) and linearly interpolated root and vertex positions. First and last frames are always kept. Shape is unchanged — you get the same F frames, some replaced by interpolated values.

Parameters:

Name	Type	Description	Default
arrays	MotionArrays	Any combination of streams; one keep-mask governs all of them.	required
drop_rate	float	Fraction of frames to drop, in [0, 1).	required
representation	str	One of "quat", "6d", "axisangle", "rotmat", "euler". Required when arrays carries joint_rot; ignored for a positions-only sample.	None
rng	numpy Generator		None
euler_orders	list of str	Required when representation="euler", ignored otherwise.	None

Returns:

Type	Description
MotionArrays

Notes

Frames 0 and F-1 are always kept, so every dropped frame has real neighbours on both sides to interpolate between. Kept frames pass through bit-identically; only the dropped ones are rebuilt.

Dropped frames are rebuilt with pybvh.rotations.quat_slerp under its shortest=True default, so for representation="quat" a rebuilt frame may come back as -q relative to the original — the same rotation in the nearer hemisphere. See :func:speed_perturbation_arrays for the full note; pybvh.rotations.quat_unwrap makes a sequence hemisphere-continuous if you need to compare components directly.

Rebuilt position frames are interpolated linearly between the kept neighbours while rotations are slerped — see :func:speed_perturbation_arrays for that divergence.