# 053. Air monitoring data availability

**Question:** Which sensor and reference fields have the weakest observation coverage?

## Result

The largest computed metric is 0.9023 for nmhc_gt; the smallest is 0.03912 for c6h6_gt. Metric: missing_share (Missing share).

Entirely blank CSV rows and unnamed empty columns were removed; missing instrument readings are retained as NaN.

![Air monitoring data availability](outputs/chart.png)

| field | observed | missing | missing_share |
| --- | --- | --- | --- |
| nmhc_gt | 914 | 8443 | 0.9023 |
| co_gt | 7674 | 1683 | 0.1799 |
| no2_gt | 7715 | 1642 | 0.1755 |
| nox_gt | 7718 | 1639 | 0.1752 |
| c6h6_gt | 8991 | 366 | 0.03912 |
| pt08_s1_co | 8991 | 366 | 0.03912 |
| pt08_s2_nmhc | 8991 | 366 | 0.03912 |
| pt08_s3_nox | 8991 | 366 | 0.03912 |
| pt08_s4_no2 | 8991 | 366 | 0.03912 |
| pt08_s5_o3 | 8991 | 366 | 0.03912 |
| t | 8991 | 366 | 0.03912 |
| rh | 8991 | 366 | 0.03912 |
| ah | 8991 | 366 | 0.03912 |

The chart shows 13 of 13 result rows; the table previews the first 13 in the analysis-defined order. [Download the full result table](outputs/results.csv). Numerical values are computed from the source; missing results stay unavailable.

## Method

Field-level availability after converting the documented -200 missing sentinel.

The study uses shared source preparation and reusable statistical routines. Its specific transformations are in [analysis.py](analysis.py), and common model/evaluation code is in [portfolio/methods.py](../../portfolio/methods.py). The [notebook](analysis.ipynb) executes the study and displays the saved results.

## Decision and limitations

Prioritize recovery of reference measurements before calibrating sensors.

The -200 sentinel is treated as missing. Reference instruments have incomplete coverage; comparisons use paired observations. Sensor calibration is retrospective, not a health or regulatory compliance assessment. 

Related studies may share observations or holdouts. These are focused analytical studies, not independent replications or deployed business systems. Any model refinements informed by these results need new untouched evaluation data. No commercial impact is inferred from an association or backtest.

## Reproduce

From the repository root, after installing `requirements.txt`:

```powershell
python projects/053-air-monitoring-data-availability/analysis.py
```

Source data are downloaded automatically if absent. Original archives are retained unchanged and checked by SHA-256. The cleaned cache normalizes column names; field-specific changes are visible in [data preparation](../../portfolio/data.py). Runtime evidence is in [receipt.json](outputs/receipt.json).

## Source

[Air Quality](https://archive.ics.uci.edu/dataset/360/air+quality), De Vito (2008). [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). Source data are transformed and aggregated in this study. [Source provenance](../../data/provenance/air.json) and [prepared-data audit](../../data/provenance/air_prepared.json) record the downloaded files, field coverage and hashes.