# 088. Fault recognition confusion analysis

**Question:** Which true fault classes are most often confused with another class?

## Result

The largest computed metric is 0.3929 for Pastry -> Other_Faults; the smallest is 0 for Dirtiness -> Pastry. Metric: actual_class_share (Share of true-class test records).

Rates use the true-class denominator; the full CSV includes zero-count off-diagonal pairs.

![Fault recognition confusion analysis](outputs/chart.png)

| actual | predicted | count | actual_class_share | confusion |
| --- | --- | --- | --- | --- |
| Bumps | Other_Faults | 33 | 0.3667 | Bumps -> Other_Faults |
| Other_Faults | Bumps | 28 | 0.2121 | Other_Faults -> Bumps |
| Pastry | Other_Faults | 11 | 0.3929 | Pastry -> Other_Faults |
| Other_Faults | Pastry | 6 | 0.04545 | Other_Faults -> Pastry |
| K_Scatch | Other_Faults | 5 | 0.05952 | K_Scatch -> Other_Faults |
| Z_Scratch | Other_Faults | 5 | 0.1471 | Z_Scratch -> Other_Faults |
| Dirtiness | Other_Faults | 3 | 0.3333 | Dirtiness -> Other_Faults |
| Bumps | Pastry | 3 | 0.03333 | Bumps -> Pastry |
| Other_Faults | Z_Scratch | 2 | 0.01515 | Other_Faults -> Z_Scratch |
| Bumps | Stains | 1 | 0.01111 | Bumps -> Stains |
| Other_Faults | Dirtiness | 1 | 0.007576 | Other_Faults -> Dirtiness |
| K_Scatch | Bumps | 1 | 0.0119 | K_Scatch -> Bumps |
| Dirtiness | Pastry | 0 | 0 | Dirtiness -> Pastry |
| Dirtiness | K_Scatch | 0 | 0 | Dirtiness -> K_Scatch |

The chart shows 14 of 42 result rows; the table previews the first 14 in the analysis-defined order. [Download the full result table](outputs/results.csv). Numerical values are computed from the source; missing results stay unavailable.

## Method

Row-normalized confusion table for the fixed forest on the grouped holdout.

The study uses shared source preparation and reusable statistical routines. Its specific transformations are in [analysis.py](analysis.py), and common model/evaluation code is in [portfolio/methods.py](../../portfolio/methods.py). The [notebook](analysis.ipynb) executes the study and displays the saved results.

## Evaluation

Fixed 80/20 split of unique input groups (seed 42); exact input duplicates stay together. Training observations: 1,552; test observations: 389. Fixed models; default decision threshold; no holdout tuning.

Target: `fault`. Features: pixels_areas, x_perimeter, y_perimeter, minimum_of_luminosity, maximum_of_luminosity, length_of_conveyer, typeofsteel_a300, typeofsteel_a400, steel_plate_thickness, edges_index, empty_index, square_index, outside_x_index, edges_x_index, edges_y_index, outside_global_index, logofareas, log_x_index, log_y_index, orientation_index, luminosity_index, sigmoidofareas. Model results and row membership are recorded in [evaluation.json](outputs/evaluation.json).

## Decision and limitations

Review recurring class confusions with inspectors before changing labels or workflow.

Every source row is a recorded fault. The data cannot estimate a production defect rate or distinguish healthy plates. Batch and machine IDs are absent, so grouped exact-input holdouts do not prove factory transfer. 

Related studies may share observations or holdouts. These are focused analytical studies, not independent replications or deployed business systems. Any model refinements informed by these results need new untouched evaluation data. No commercial impact is inferred from an association or backtest.

## Reproduce

From the repository root, after installing `requirements.txt`:

```powershell
python projects/088-fault-recognition-confusion-analysis/analysis.py
```

Source data are downloaded automatically if absent. Original archives are retained unchanged and checked by SHA-256. The cleaned cache normalizes column names; field-specific changes are visible in [data preparation](../../portfolio/data.py). Runtime evidence is in [receipt.json](outputs/receipt.json).

## Source

[Steel Plates Faults](https://archive.ics.uci.edu/dataset/198/steel+plates+faults), Buscema, Terzi and Tastle (2010). [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Source data are transformed and aggregated in this study. [Source provenance](../../data/provenance/steel.json) and [prepared-data audit](../../data/provenance/steel_prepared.json) record the downloaded files, field coverage and hashes.