GMS SKU Decoder — Skill Eval Review

Iteration 1 • 2026-03-28 • 3 evals, 18 assertions • Claude Sonnet executor + Claude Opus grader

100%
With Skill (18/18)
44%
Without Skill (8/18)
+56%
Skill Improvement

Eval 1: Decode Tricky PIDs

With: 7/7 Without: 2/7
With Skill
PASS CRD810 = Crown Drip 8x10 (not CRD81 + gauge 0)
Correctly identifies CRD810 as family prefix with embedded '810'
PASS CO4129ARW = 41" coil, 29ga, Arctic White
Decodes: CO=coil, 41=width, 29=gauge, ARW=Arctic White
PASS CRRC10 = Crown Ridge Cap 10in
Identifies CRRC10 as family prefix, not CRRC + gauge 1
PASS SSQ675 24ga, CMGAC = standard finish (NOT crinkle)
CMGAC = CMG Aged Copper, standard (1.00x). C is part of code, not crinkle suffix.
PASS MCC = matte finish (+8%), not crinkle (+15%)
States "Matte (1.08x premium)" — correctly distinguishes from crinkle
PASS CRBK = Crinkle Black (+15%)
Identifies CR prefix as crinkle with +15% multiplier
PASS A9AG = Ag Panel 29ga (gauge in family name)
Explicitly notes "gauge embedded in family prefix, no separate digit"
Without Skill
FAIL CRD810 = Crown Drip 8x10
Treats '8109' as style code, misidentifies gauge
FAIL CO4129ARW = 41" coil, 29ga
Doesn't decode 41" width or 29ga from digits
FAIL CRRC10 = Crown Ridge Cap 10in
Treats '109' as family code, wrong gauge
FAIL CMGAC = standard, NOT crinkle
Uncertain on gauge, doesn't confirm non-crinkle
FAIL MCC = matte (+8%)
Says MCC = +15% premium. WRONG — MCC is matte (+8%)
PASS CRBK = Crinkle (+15%)
Correctly identifies CR as crinkle prefix
FAIL A9AG = gauge in family name
Says "Gauge: Not encoded in ID" — misses A9 = 29ga

Eval 2: Encode Products from Description

With: 5/5 Without: 3/5
With Skill
PASS Drip Edge → DE6AG12
Exact match: DE + 6 + AG + 12
PASS Ridge Cap 10" uses RC10 family prefix
Generates RC109ARW10, correctly uses RC10 not RC
PASS Pro Panel has NO length code
Generates PRO9BK — panels don't have length
PASS Matte Black = MBB color code
Correct: MBB (not BK or BKM)
PASS 24ga CMG = gauge digit 4 + CMG color
Generates SSQ6754CMCG — valid format
Without Skill
FAIL Drip Edge → DE6AG12
Generates DE-26-AG-12FT (hyphens, FT suffix — wrong format)
FAIL Ridge Cap 10" uses RC10
Format RC10-29-ARW-10FT (hyphens, wrong format)
PASS Pro Panel has NO length code
Correctly omits length (though uses PROPAN prefix)
FAIL Matte Black = MBB
Uses BKM instead of MBB — wrong code order
FAIL 24ga CMG encoding
Uses SSQ675-24-CMCG (hyphens, '24' not digit '4')

Eval 3: Material Sourcing & Product Categories

With: 6/6 Without: 3/6
With Skill
PASS RC20 = Pattern B (41"), source CO4129ARW
Exact coil PID and pattern identification
PASS HFRG = 43" special, source CO4326AG
Correctly identifies the 43" exception for HFRG
PASS PCF = Pattern A (20"), source CO2029BK
Standard 20" coil sourcing for trim
PASS SSQ550 = Hidden Fastener, 40% margin
Correct category and margin target
PASS PRO = Exposed Fastener, 35% margin
Correct: 35% (not 40%)
PASS RC20 uses 41" due to stretchout
Explains: RC20 stretchout = 21.0"/ft vs PCF's lower stretchout
Without Skill
FAIL RC20 = Pattern B, CO4129ARW
Vague: mentions 41" but doesn't name CO4129ARW source
FAIL HFRG = 43" special, CO4326AG
Says "41-inch material" — misses the 43" special exception
PASS PCF = Pattern A (20")
Correctly identifies 20" material
PASS SSQ550 = Hidden Fastener, 40%
Correct margin (general panel knowledge)
FAIL PRO = Exposed Fastener, 35%
Says 40% — wrong, PRO is 35%
FAIL RC20 stretchout explanation
General "broader coverage" — misses stretchout metric

Key Findings

Embedded Numbers: Without the skill, Claude treats CRD810/CRRC10 embedded numbers as gauge digits. The skill's greedy longest-first matching prevents this.
Coil Format: CO4129ARW uses 2-digit width (41) + 2-digit gauge (29) + color. Without the skill, Claude can't parse this non-obvious structure.
Matte vs Crinkle: The CLAUDE.md glossary says "MCC/CR/ULG = premium (+15%)" but MCC is actually matte (+8%). The skill corrects this critical error.
PID Format: GMS uses no-separator format (DE6AG12). Without skill, Claude defaults to hyphenated format (DE-26-AG-12FT) common in other metal systems.
HFRG Exception: HFRG is the ONLY family using 43" material. Without the skill, Claude lumps it with standard 41" Pattern B families.
Color Code Order: Matte Black = MBB (M prefix + BB), not BKM (BK suffix + M). Without the skill, Claude guesses the wrong code construction.