mptyl/ThothII
1
0
Fork 0
Code Issues 25 Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Update interactive presentation and presenter console

Browse Source
This commit is contained in:
Codex
2026-09-20 19:43:33 +02:00
parent 348f4a9738
commit 5217c574e9
21 changed files with 1318 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files
presentation/slides/08-validation-sources.md
+69
View File
@@ -0,0 +1,69 @@
# Slide 08: validation sources
Verified on 2026-09-20 against `aritmolab/chirone-etl` on the hospital Gitea,
main snapshot `2ab00106188f298c1a0c1e2c48b3a4dbc66c45b5`. These are source-code
and documentation checks, not a new audit of patient data or a production test run.
## Quality gates
[Extraction specification, SC-003 and SC-004](https://git-aritmolab.policlinicosandonato.it/aritmolab/chirone-etl/src/commit/2ab00106188f298c1a0c1e2c48b3a4dbc66c45b5/include/textanalysis/specs/001-cr-conclusioni-extraction/spec.md)
defines procedure classification accuracy of at least 95% against a random sample
of 100 manually reviewed records. The separate 85% target is coverage: at least one
recognized arrhythmia in that proportion of records, assuming most procedures
concern known arrhythmias. It is not diagnostic accuracy, sensitivity or precision.
The repository search also confirms these definitions in the TextAnalysis README.
The inspected sources do not establish measured results or a completed 100-record
review for every release. The slide therefore calls these figures targets.
## Clinical criteria and cohort selection
[Confirmed-diagnosis mart documentation](https://git-aritmolab.policlinicosandonato.it/aritmolab/chirone-etl/src/commit/2ab00106188f298c1a0c1e2c48b3a4dbc66c45b5/docs/marts/procedure-event-marts/mart_diagnosi_lettera_confermata.md)
identifies the Superset “Diagnosi Lettere” charts (570/571), in the “Test Provocativi
& Diagnosi Lettere” dashboard, as consumers of this dataset.
[The SQL implementation](https://git-aritmolab.policlinicosandonato.it/aritmolab/chirone-etl/src/commit/2ab00106188f298c1a0c1e2c48b3a4dbc66c45b5/dbt/models/marts/mart_diagnosi_lettera_confermata.sql)
first requires patient attribution and a non-negated finding. For Brugada and
long QT syndrome it additionally requires a positive provocative test for that
condition OR an ablation recorded for that pathology, at patient level. It does
not impose that extra condition on other pathologies. These are operational
cohort-selection rules, not a universal clinical diagnostic standard.
No formal sign-off or individual clinical validation of the entire Brugada cohort
was established by the consulted sources. The point is titled “Clinical criteria”:
the queries apply the selection rules and Superset displays their result. The
speaker script explains that a disease mention alone is insufficient for cohort
inclusion; the popup retains the Brugada/long QT rule as a project-specific example.
## Feedback loop: the approximately 893 records
[Known limits and version changelog](https://git-aritmolab.policlinicosandonato.it/aritmolab/chirone-etl/src/commit/2ab00106188f298c1a0c1e2c48b3a4dbc66c45b5/docs/architecture/textanalysis-known-limits.md)
records an audit on 2026-06-11 and the v1.3.1 correction on 2026-06-12. It reports
approximately 893 Brugada false positives in the newer letter format (8.4% of
non-negated Brugada records). This is a record count, not a distinct-patient count.
[Regression tests, class TestNegationExtended](https://git-aritmolab.policlinicosandonato.it/aritmolab/chirone-etl/src/commit/2ab00106188f298c1a0c1e2c48b3a4dbc66c45b5/include/textanalysis/tests/unit/test_letter_cascade_analyzer.py)
explain the defect: the negation patterns recognized “negato” and “negata”, but
omitted “negativo”. The fix adds `negativ[oai]` to both pre- and post-context
negation detection. The test sentence “Test alla flecainide negativo per sindrome
di Brugada” must still yield a Brugada finding, now with `negato=True`; a separate
test preserves the affirmed classification of a positive Brugada diagnosis.
Thus “reclassified” is more accurate than “removed”. These tests were inspected,
not executed in this presentation workspace.
## Retained context and limits
The mart documentation describes propagation of `attribuzione` and `negato`
through integration, warehouse bridge and the mentions mart. The confirmed view
filters them without deleting the underlying mentions dataset.
The known-limits document describes a context window of up to 60 characters on
either side of a match, bounded by a full stop, semicolon or newline. Uncertainty
phrases such as “sospetto di” and “per escludere” are not automatically definitive
negations. The same-pathology deduplication tests show that an affirmed patient
finding takes priority over a negated or family finding. Therefore the slide does
not claim verbatim preservation of every repeated match or complete extraction
of all clinical information.
The mart page's historical “first matching field wins” caveat predates the v1.3.0
multi-field union. For current field handling, the later known-limits document
and unit tests take precedence.