UpstreamDrift Learning, RL, and Analytics
Plain-Language Summary
This reference catalogs UpstreamDrift learning and analytics modules at a pinned commit: hole MDP, shot-pattern scoring, RL Gymnasium environments, launch-monitor import, imitation scaffolding, and domain randomization. It records defaults and limits without prescribing training recipes.
Key Takeaways
- With green-terminal preprocessing, hole MDP value iteration reports strokes-to-green; putting is not included.
- Gymnasium environments expose configurable observation blocks and default torque actions without an in-repo PPO bundle.
- Launch import maps columns with profile confidence below 0.5 falling back to a generic profile.
- Shipped GAIL scores policy actions with noise on expert states rather than full on-policy rollouts.
Implementation reference for learning and analytics modules at UpstreamDrift 3afe09c1. Paths are under the UpstreamDrift repository unless noted.
Strokes-Gained Hole MDP (sg_optimizer)
Modules: src/shared/python/sg_optimizer/mdp/value_iteration.py, course/rasterize.py, shot_model/distributions.py, data/sg_optimizer/baselines/pga_tour.yaml.
State, Grid, and “Expected Strokes”
- Course lies are rasterized to integer codes
LIE_CODES: tee=0, fairway=1, rough=2, trees=3, sand=4, water=5, ob=6, green=7, holed=8 (rasterize.py). - In the raw
HoleMDP, onlyholedcells are terminal. The optionalgreen_terminal()helper copies the raster and relabels every green cell asholed, making all those cells terminal for every action. The solver does not callgreen_terminal()automatically; callers must choose the raster explicitly. - With that preprocessing,
HoleMDPvalue iteration estimates strokes-to-green from each cell under a player dispersion profile andpga_tour.yamldispersion baseline; putting is not included. The terminal label does not supply a Broadie strokes-to-hole-out boundary value. Dispersion usesTiltedBivariateGaussian: \(\Delta_\|\) past target, \(\Delta_\perp\) left-positive, correlation \(\rho\). - Bellman backup (vectorized): for each cell, sample
n_samplesshot outcomes (fixed common random numbers across sweeps), cost \(1 + \text{extra} + V(s')\); hazards (water/OB/off-grid) cost \(2 + V(s)\): one stroke for the shot, one penalty stroke, then continuation from the current cell (value_iteration.py~371–373). - Aim angles are pin-relative (0° points at the pin from each cell).
The terminal contract is explicit in pinned green_terminal() and the solver’s _terminal_mask. For example, after relabelling, \(V(s_{\mathrm{green}})=0\), so a deterministic shot reaching a green cell has cost \(1+V(s_{\mathrm{green}})=1\). If a separate putting model assigned 1.8 expected putts at that landing point, a hole-out objective would count 2.8 expected strokes for that shot and continuation. That 1.8 is an illustrative value, not a shipped constant. A putting boundary that varies with landing position can also change the preferred approach; adding one constant after solving does not generally recover the hole-out optimal policy. The shot-pattern scoring kernel below is a separate model and is not automatically used as this MDP’s terminal cost.
Numbers flagged # ESTIMATE: in pga_tour.yaml (e.g. per-club \(\rho\)) are not direct Broadie tabulations.
See also On the Limits of Strokes-Gained Inference for population vs optimisation semantics.
Shot-Pattern Scoring Kernel
Module: src/tools/shot_pattern_analysis/scoring.py.
Putting on the green uses Broadie Eq. 7 form \(E_{\mathrm{green}} = 2 - p_1 + p_3\) with:
| Quantity | Value in code |
|---|---|
| \(\sigma_{\mathrm{angle}}\) | \(1.46^\circ\) |
| \(\sigma_{\mathrm{distance}}\) | \(0.057\) (yd scaling in denominator) |
| \(p_3\) logistic | \(1/(1+\exp(5.49 - 0.106\cdot\mathrm{ft} + 0.000563\cdot\mathrm{ft}^2)) - 0.00398\) |
| Table 9 approach range | 10–600 yd piecewise-linear tables in-file |
The three-putt curve is an anchor-reconciled approximation (docstring notes Eq. 6 coefficient ordering issues in the paper). Approach tables cite Broadie (2011) PDF SHA pinned in SOURCE_PDF_SHA256.
The AffineDrift simulation article Straight, Draw, and Fade adds a methods box below with these equations for cross-checking reported SG deltas.
RL Environments
Modules: src/learning/rl/humanoid_envs.py, configs.py; base class base_env.py.
| Env | Termination (defaults) | Reward structure |
|---|---|---|
HumanoidWalkEnv |
base_height < 0.3 m; tilt proxy up_z < 0.8 from quaternion \((w,x,y,z)\) |
\(\exp(-\|\mathbf{v}_{xy}-\mathbf{v}_{\mathrm{target}}\|)\) weighted + alive + raw \(\|\boldsymbol{\tau}\|^2\) energy penalty |
HumanoidStandEnv |
Similar balance thresholds | Posture + energy (see module) |
Observation: concatenation of enabled blocks from ObservationConfig.get_obs_dim—defaults include joint pos/vel only (\(2 n_{\mathrm{joints}}\)). IMU adds 6 (3 accel + 3 gyro) if enabled.
Action: joint torques by default (ActionMode.TORQUE).
Trainer: no PPO/SB3 bundle in-repo; attach an external trainer to the Gymnasium API.
Tilt test uses \(up_z = 1 - 2(x^2 + y^2)\) with scalar-first quaternion order from get_base_orientation().
Domain Randomization Defaults
Module: src/learning/sim2real/domain_randomization.py.
DomainRandomizationConfig applies independent uniform multipliers per family on main:
| Parameter | Default range |
|---|---|
| mass | 0.8–1.2× |
| friction | 0.5–1.5× |
| damping | 0.8–1.2× |
| motor | 0.9–1.1× |
| gravity magnitude | 9.5–10.1 m/s² with randomize_gravity=True |
Geometry of Motion Volume 0, Machine Learning now carries a boxed note reconciling this with the textbook guidance on correlated mass/inertia perturbations.
Launch-Monitor Import Contract
Canonical layer: Tools src/shared/python/launch_monitor/ (imported via src/tools/launch_monitor_model).
- 33 canonical metrics in
METRICS(schema.py); storage SI (m, m/s, rad, rad/s). - Default display unit map: lengths default to yards when header and mapping lack units (
profiles.py_defaults()). - Unit resolution order: explicit
ColumnMapping.source_unit→ parsed header → profile default (importer.py~247–257). - Profile detection: signature match score;
confidence < 0.5→ generic profile. - No per-vendor sign normalisation beyond unit conversion—vendor conventions in Launch Monitor Vendor Reference are not auto-corrected at import.
Neural Proposal vs Chapter Families
| Module | What ships |
|---|---|
src/shared/python/neural_motion/proposals/model.py |
Time-mean masked trajectory features → MLP → tanh×CONTROL_SCALE heads; mixture + mode bias |
motion_matching.inverse + training/runtime/adapters/pytorch_cvae.py |
CVAE proposal path (separate from neural_motion) |
Forward Surrogates and Multimodal Inverse Proposals includes a repository status table.
Imitation Stack (Simplified GAIL)
Modules: src/learning/imitation/_bc.py, _dagger.py, _gail.py.
Shipped GAIL uses a discriminator on \((s,a)\) pairs; policy actions for the discriminator are generated with Gaussian exploration noise on expert states without environment rollouts (_gail.py ~237–238). Treat it as a teaching scaffold, not full generative adversarial imitation with on-policy sampling.
Tour Baselines Naming
| Package | Purpose |
|---|---|
src/shared/python/tour_baselines/ |
Mocap swing / Matched Swing Program capture baselines |
data/sg_optimizer/baselines/pga_tour.yaml |
Dispersion + scoring baselines for the hole MDP |
Do not interchange these when auditing expected-strokes numbers.
Citation
@online{olson2026,
author = {Olson, Dieter},
title = {UpstreamDrift {Learning,} {RL,} and {Analytics}},
date = {2026-11-11},
url = {https://affinedrift.com/articles/upstreamdrift-learning-analytics.html},
langid = {en},
abstract = {Implementation map for UpstreamDrift learning,
reinforcement-learning environments, and analytics at commit
3afe09c1. Covers strokes-gained hole value iteration, shot-pattern
scoring constants, Gymnasium observation and action shapes,
domain-randomization multipliers, launch-monitor profile detection,
neural proposal versus CVAE paths, and simplified GAIL. Emphasizes
what is implemented versus external trainer attachment and
teaching-scaffold limitations.}
}