Skip to main content

UpstreamDrift Learning, RL, and Analytics

reference
tools
SG hole MDP, shot-pattern scoring, RL environments, launch-monitor import, imitation stack, and domain-randomization defaults.
Author

Dieter Olson

Published

November 11, 2026

Modified

November 11, 2026

Abstract
Implementation map for UpstreamDrift learning, reinforcement-learning environments, and analytics at commit 3afe09c1. Covers strokes-gained hole value iteration, shot-pattern scoring constants, Gymnasium observation and action shapes, domain-randomization multipliers, launch-monitor profile detection, neural proposal versus CVAE paths, and simplified GAIL. Emphasizes what is implemented versus external trainer attachment and teaching-scaffold limitations.

Plain-Language Summary

This reference catalogs UpstreamDrift learning and analytics modules at a pinned commit: hole MDP, shot-pattern scoring, RL Gymnasium environments, launch-monitor import, imitation scaffolding, and domain randomization. It records defaults and limits without prescribing training recipes.

Key Takeaways

  • With green-terminal preprocessing, hole MDP value iteration reports strokes-to-green; putting is not included.
  • Gymnasium environments expose configurable observation blocks and default torque actions without an in-repo PPO bundle.
  • Launch import maps columns with profile confidence below 0.5 falling back to a generic profile.
  • Shipped GAIL scores policy actions with noise on expert states rather than full on-policy rollouts.

Implementation reference for learning and analytics modules at UpstreamDrift 3afe09c1. Paths are under the UpstreamDrift repository unless noted.

Strokes-Gained Hole MDP (sg_optimizer)

Modules: src/shared/python/sg_optimizer/mdp/value_iteration.py, course/rasterize.py, shot_model/distributions.py, data/sg_optimizer/baselines/pga_tour.yaml.

State, Grid, and “Expected Strokes”

  • Course lies are rasterized to integer codes LIE_CODES: tee=0, fairway=1, rough=2, trees=3, sand=4, water=5, ob=6, green=7, holed=8 (rasterize.py).
  • In the raw HoleMDP, only holed cells are terminal. The optional green_terminal() helper copies the raster and relabels every green cell as holed, making all those cells terminal for every action. The solver does not call green_terminal() automatically; callers must choose the raster explicitly.
  • With that preprocessing, HoleMDP value iteration estimates strokes-to-green from each cell under a player dispersion profile and pga_tour.yaml dispersion baseline; putting is not included. The terminal label does not supply a Broadie strokes-to-hole-out boundary value. Dispersion uses TiltedBivariateGaussian: \(\Delta_\|\) past target, \(\Delta_\perp\) left-positive, correlation \(\rho\).
  • Bellman backup (vectorized): for each cell, sample n_samples shot outcomes (fixed common random numbers across sweeps), cost \(1 + \text{extra} + V(s')\); hazards (water/OB/off-grid) cost \(2 + V(s)\): one stroke for the shot, one penalty stroke, then continuation from the current cell (value_iteration.py ~371–373).
  • Aim angles are pin-relative (0° points at the pin from each cell).

The terminal contract is explicit in pinned green_terminal() and the solver’s _terminal_mask. For example, after relabelling, \(V(s_{\mathrm{green}})=0\), so a deterministic shot reaching a green cell has cost \(1+V(s_{\mathrm{green}})=1\). If a separate putting model assigned 1.8 expected putts at that landing point, a hole-out objective would count 2.8 expected strokes for that shot and continuation. That 1.8 is an illustrative value, not a shipped constant. A putting boundary that varies with landing position can also change the preferred approach; adding one constant after solving does not generally recover the hole-out optimal policy. The shot-pattern scoring kernel below is a separate model and is not automatically used as this MDP’s terminal cost.

Numbers flagged # ESTIMATE: in pga_tour.yaml (e.g. per-club \(\rho\)) are not direct Broadie tabulations.

See also On the Limits of Strokes-Gained Inference for population vs optimisation semantics.

Shot-Pattern Scoring Kernel

Module: src/tools/shot_pattern_analysis/scoring.py.

Putting on the green uses Broadie Eq. 7 form \(E_{\mathrm{green}} = 2 - p_1 + p_3\) with:

Quantity Value in code
\(\sigma_{\mathrm{angle}}\) \(1.46^\circ\)
\(\sigma_{\mathrm{distance}}\) \(0.057\) (yd scaling in denominator)
\(p_3\) logistic \(1/(1+\exp(5.49 - 0.106\cdot\mathrm{ft} + 0.000563\cdot\mathrm{ft}^2)) - 0.00398\)
Table 9 approach range 10–600 yd piecewise-linear tables in-file

The three-putt curve is an anchor-reconciled approximation (docstring notes Eq. 6 coefficient ordering issues in the paper). Approach tables cite Broadie (2011) PDF SHA pinned in SOURCE_PDF_SHA256.

The AffineDrift simulation article Straight, Draw, and Fade adds a methods box below with these equations for cross-checking reported SG deltas.

RL Environments

Modules: src/learning/rl/humanoid_envs.py, configs.py; base class base_env.py.

Env Termination (defaults) Reward structure
HumanoidWalkEnv base_height < 0.3 m; tilt proxy up_z < 0.8 from quaternion \((w,x,y,z)\) \(\exp(-\|\mathbf{v}_{xy}-\mathbf{v}_{\mathrm{target}}\|)\) weighted + alive + raw \(\|\boldsymbol{\tau}\|^2\) energy penalty
HumanoidStandEnv Similar balance thresholds Posture + energy (see module)

Observation: concatenation of enabled blocks from ObservationConfig.get_obs_dim—defaults include joint pos/vel only (\(2 n_{\mathrm{joints}}\)). IMU adds 6 (3 accel + 3 gyro) if enabled.

Action: joint torques by default (ActionMode.TORQUE).

Trainer: no PPO/SB3 bundle in-repo; attach an external trainer to the Gymnasium API.

Tilt test uses \(up_z = 1 - 2(x^2 + y^2)\) with scalar-first quaternion order from get_base_orientation().

Domain Randomization Defaults

Module: src/learning/sim2real/domain_randomization.py.

DomainRandomizationConfig applies independent uniform multipliers per family on main:

Parameter Default range
mass 0.8–1.2×
friction 0.5–1.5×
damping 0.8–1.2×
motor 0.9–1.1×
gravity magnitude 9.5–10.1 m/s² with randomize_gravity=True

Geometry of Motion Volume 0, Machine Learning now carries a boxed note reconciling this with the textbook guidance on correlated mass/inertia perturbations.

Launch-Monitor Import Contract

Canonical layer: Tools src/shared/python/launch_monitor/ (imported via src/tools/launch_monitor_model).

  • 33 canonical metrics in METRICS (schema.py); storage SI (m, m/s, rad, rad/s).
  • Default display unit map: lengths default to yards when header and mapping lack units (profiles.py _defaults()).
  • Unit resolution order: explicit ColumnMapping.source_unit → parsed header → profile default (importer.py ~247–257).
  • Profile detection: signature match score; confidence < 0.5 → generic profile.
  • No per-vendor sign normalisation beyond unit conversion—vendor conventions in Launch Monitor Vendor Reference are not auto-corrected at import.

Neural Proposal vs Chapter Families

Module What ships
src/shared/python/neural_motion/proposals/model.py Time-mean masked trajectory features → MLP → tanh×CONTROL_SCALE heads; mixture + mode bias
motion_matching.inverse + training/runtime/adapters/pytorch_cvae.py CVAE proposal path (separate from neural_motion)

Forward Surrogates and Multimodal Inverse Proposals includes a repository status table.

Imitation Stack (Simplified GAIL)

Modules: src/learning/imitation/_bc.py, _dagger.py, _gail.py.

Shipped GAIL uses a discriminator on \((s,a)\) pairs; policy actions for the discriminator are generated with Gaussian exploration noise on expert states without environment rollouts (_gail.py ~237–238). Treat it as a teaching scaffold, not full generative adversarial imitation with on-policy sampling.

Tour Baselines Naming

Package Purpose
src/shared/python/tour_baselines/ Mocap swing / Matched Swing Program capture baselines
data/sg_optimizer/baselines/pga_tour.yaml Dispersion + scoring baselines for the hole MDP

Do not interchange these when auditing expected-strokes numbers.

Citation

BibTeX citation: Download .bib
@online{olson2026,
  author = {Olson, Dieter},
  title = {UpstreamDrift {Learning,} {RL,} and {Analytics}},
  date = {2026-11-11},
  url = {https://affinedrift.com/articles/upstreamdrift-learning-analytics.html},
  langid = {en},
  abstract = {Implementation map for UpstreamDrift learning,
    reinforcement-learning environments, and analytics at commit
    3afe09c1. Covers strokes-gained hole value iteration, shot-pattern
    scoring constants, Gymnasium observation and action shapes,
    domain-randomization multipliers, launch-monitor profile detection,
    neural proposal versus CVAE paths, and simplified GAIL. Emphasizes
    what is implemented versus external trainer attachment and
    teaching-scaffold limitations.}
}
For attribution, please cite this work as:
Olson, Dieter. 2026. “UpstreamDrift Learning, RL, and Analytics.” November 11. https://affinedrift.com/articles/upstreamdrift-learning-analytics.html.