Multimodal AI · Case study
Hate & Threat Detection in Forensics
A reproducible forensic pipeline that routes image evidence, OCR text, and associated language through specialized models before creating an auditable case-level risk assessment.
- Context
- Multimodal digital forensics
- Role
- Researcher and developer
- Evidence
- Image, OCR, and associated text
- Output
- Auditable case-level assessment
- Status
- Reproducible research system
The problem
Forensic meaning is often split across several evidence sources
A single image can contain visual symbols, embedded text, and surrounding language that change the interpretation of a case. A single-model decision can miss this context and make it difficult for an analyst to understand which evidence influenced the result.
The system
Specialized analysis before transparent score fusion
Each evidence source is handled by a model suited to that modality. The resulting signals are joined at the score level and preserved with source information for analyst review.
What I built
A reproducible pipeline built for analyst inspection
Created source-aware routing for visual evidence, OCR-derived text, and associated language.
Applied OpenCLIP-based visual scoring and DeBERTa-based language analysis within one controlled workflow.
Implemented score-level fusion across multiple evidence signals without hiding the contributing sources.
Added CSV evidence logging so cases, model outputs, and fused assessments can be inspected after a run.
Structured the project for command-line execution, repeatable experiments, and regression testing.
Documented the research system and connected the public implementation to the paper.
Quality and evaluation
Traceability supports both error analysis and responsible use
The workflow keeps modality-specific outputs available for error analysis instead of presenting only a final label.
The system is research software that supports forensic review. Its assessments require human interpretation and should not be treated as autonomous legal or investigative conclusions.
Technology
Evidence