# 021. Call duration leakage audit

**Question:** How much does unavailable call-duration information change retrospective model scores?

## Result

The largest computed metric is 0.5529 for Includes completed duration / Logistic regression; the smallest is 0.3158 for Pre-call / Prior baseline. Metric: average_precision (Average precision).

The duration variant is intentionally invalid for pre-call use. Any score increase illustrates an information-timing problem.

![Call duration leakage audit](outputs/chart.png)

| model | accuracy | balanced_accuracy | macro_f1 | roc_auc | average_precision | brier | information | variant |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Prior baseline | 0.6842 | 0.5 | 0.4062 | 0.5 | 0.3158 | 0.2779 | Pre-call | Pre-call / Prior baseline |
| Logistic regression | 0.6806 | 0.5272 | 0.4906 | 0.6632 | 0.4818 | 0.2353 | Pre-call | Pre-call / Logistic regression |
| Random forest | 0.683 | 0.5036 | 0.4214 | 0.674 | 0.4641 | 0.2365 | Pre-call | Pre-call / Random forest |
| Prior baseline | 0.6842 | 0.5 | 0.4062 | 0.5 | 0.3158 | 0.2779 | Includes completed duration | Includes completed duration / Prior baseline |
| Logistic regression | 0.7028 | 0.5696 | 0.5587 | 0.7501 | 0.5529 | 0.2235 | Includes completed duration | Includes completed duration / Logistic regression |
| Random forest | 0.6941 | 0.5267 | 0.4721 | 0.7649 | 0.5525 | 0.2253 | Includes completed duration | Includes completed duration / Random forest |

The chart shows 6 of 6 result rows; the table previews the first 6 in the analysis-defined order. [Download the full result table](outputs/results.csv). Numerical values are computed from the source; missing results stay unavailable.

## Method

Paired comparison with identical chronological test rows and duration included only in the diagnostic variant.

The study uses shared source preparation and reusable statistical routines. Its specific transformations are in [analysis.py](analysis.py), and common model/evaluation code is in [portfolio/methods.py](../../portfolio/methods.py). The [notebook](analysis.ipynb) executes the study and displays the saved results.

## Evaluation

First 80% of ordered observations train; final 20% test. Training observations: 36,168; test observations: 9,043. Fixed models; default decision threshold; no holdout tuning.

Target: `subscribed`. Features: contact, month, day, campaign, pdays, previous, poutcome. Model results and row membership are recorded in [evaluation.json](outputs/evaluation.json).

## Decision and limitations

Reject duration from any pre-call model even if the retrospective score rises.

Observational marketing records do not identify campaign uplift. Month and row order are not precise timestamps; repeated-client identifiers are unavailable. Models are research diagnostics, not financial eligibility or automated contact decisions. 

Related studies may share observations or holdouts. These are focused analytical studies, not independent replications or deployed business systems. Any model refinements informed by these results need new untouched evaluation data. No commercial impact is inferred from an association or backtest.

## Reproduce

From the repository root, after installing `requirements.txt`:

```powershell
python projects/021-call-duration-leakage-audit/analysis.py
```

Source data are downloaded automatically if absent. Original archives are retained unchanged and checked by SHA-256. The cleaned cache normalizes column names; field-specific changes are visible in [data preparation](../../portfolio/data.py). Runtime evidence is in [receipt.json](outputs/receipt.json).

## Source

[Bank Marketing](https://archive.ics.uci.edu/dataset/222/bank+marketing), Moro, Rita and Cortez (2014). [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Source data are transformed and aggregated in this study. [Source provenance](../../data/provenance/bank.json) and [prepared-data audit](../../data/provenance/bank_prepared.json) record the downloaded files, field coverage and hashes.