HierCore: Multi-class Image Anomaly Detection for Practical Applications

Official implementation of "Multi-class Image Anomaly Detection for Practical Applications: Requirements and Robust Solutions" accepted at Neurocomputing (2026).

📋 Abstract

Recent advances in image anomaly detection have expanded unsupervised models from single-class scenarios to multi-class frameworks, mainly to reduce training cost and model storage. However, when a single model simultaneously handles multiple classes, its per-class detection accuracy often falls short compared to class-specific approaches. Accordingly, most prior work has focused on closing this accuracy gap. Despite this effort, one important factor remains overlooked: how the availability of class information influences anomaly detection, particularly in defining detection thresholds. Whether thresholds are class-specific or class-agnostic, this choice plays a critical role in multi-class image anomaly detection.

In this study, we identify and formalize the essential requirements that multi-class unsupervised anomaly detection models must satisfy under both known-class and unknown-class evaluation settings. We then re-evaluate existing approaches under these requirements and reveal their limitations. To address these challenges, we propose Hierarchical Coreset (HierCore), a novel framework that can operate without class labels by leveraging a hierarchical memory bank to infer class-wise decision criteria. We conduct extensive experiments to assess the robustness and applicability of both prior methods and our own under varying label conditions. The results show that HierCore consistently satisfies all defined requirements and delivers stable, high detection performance across settings, demonstrating its strong potential for real-world multi-class anomaly detection.

🏗️ Architecture

Motivation

We define two essential requirements for applying Multi-Class Unsupervised Image Anomaly Detection (MC-UIAD) in real-world scenarios:

1. Minimizing performance disparities across classes in the One-Class UIAD (OC-UIAD) setting
2. Ensuring robust detection performance even under unknown-class conditions

HierCore Framework

Hierarchical Coreset (HierCore) is a novel MC-UIAD framework designed to remain effective under varying levels of label availability. HierCore operates reliably even without class labels by:

1. Semantic Clustering: Leverages semantic information extracted from deep layers (e.g., layer4) to group data and estimate latent class groupings using FINCH clustering with silhouette score for optimal partition selection
2. Hierarchical Memory Bank: Constructs cluster-specific memory banks using normal samples within each semantic cluster, with cluster-specific keys for efficient matching
3. Cluster-wise Inference: During inference, new samples are matched with their most relevant semantic cluster, and anomalies are detected using cluster-specific thresholds

Main Contributions

• Requirement Definition: Define two requirements that a model must satisfy to enable practical application of multi-class unsupervised image anomaly detection
• Systematic Re-evaluation: Re-evaluate existing MC-UIAD methods under four realistic scenarios to assess whether they meet the defined requirements, using four industrial benchmark datasets
• Robust Framework: Propose HierCore, which remains effective across all scenarios including those where class labels are unavailable

📦 Installation

Requirements

• Docker image: nvcr.io/nvidia/pytorch:22.12-py3 (recommended)
• Python 3.8+
• CUDA-compatible GPU

Setup

# Clone the repository
git clone https://github.com/your-repo/HierCore.git
cd HierCore

# Install dependencies
pip install -r requirements.txt

Dependencies

faiss-gpu==1.7.2
timm==0.9.12
omegaconf==2.3.0
scikit-image
ADEval==1.1.0
finch-clust==0.2.0

📂 Dataset Preparation

Download and organize the datasets as follows:

/datasets/
├── MVTec/
│   ├── bottle/
│   ├── cable/
│   └── ...
├── VisA/
│   ├── candle/
│   ├── capsules/
│   └── ...
├── BTAD/
│   ├── 01/
│   ├── 02/
│   └── 03/
└── MPDD/
    ├── bracket_black/
    ├── bracket_brown/
    └── ...

Supported Datasets

Dataset	Classes	Description
MVTec AD	15	Industrial anomaly detection benchmark
VisA	12	Visual anomaly detection dataset
BTAD	3	BeanTech Anomaly Detection dataset
MPDD	6	Metal Parts Defect Detection dataset

🚀 Usage

Training & Evaluation

Single Dataset

# MVTec AD - HierarchicalPatchCore (Multi-class)
python main.py config=./configs.yaml \
    DATASET.classname=all \
    DATASET.name=MVTec \
    DATASET.datadir=/datasets/MVTec \
    MODEL.name=HierarchicalPatchCore \
    MODEL.sampler=greedy_coreset \
    MODEL.sampler_ratio=0.1 \
    MODEL.semantic_layer=layer4 \
    MODEL.semantic_batch_size=32 \
    DEFAULT.savedir=./results/MVTec/HierarchicalPatchCore/greedy0.1-layer4 \
    DATASET.resize=288 \
    DATASET.imagesize=256

Run All Experiments

# Run all dataset experiments
bash ./runs/run.sh

# Or run individual datasets
bash ./runs/run_mvtec.sh
bash ./runs/run_visa.sh
bash ./runs/run_btad.sh
bash ./runs/run_mpdd.sh

Key Configuration Options

Parameter	Description	Default
MODEL.name	Model type (PatchCore, HierarchicalPatchCore)	PatchCore
MODEL.semantic_layer	Layer for semantic feature extraction	layer4
MODEL.clustering_method	Clustering algorithm (finch, kmeans, dbscan)	finch
MODEL.sampler_ratio	Coreset sampling ratio	0.1
MODEL.known_class	Use ground-truth class labels	false
MODEL.n_clusters	Number of clusters for K-means	Dataset-specific
DATASET.classname	Class name or all for multi-class	-

Baseline Comparison (PatchCore)

# Single-class PatchCore
python main.py config=./configs.yaml \
    DATASET.classname=bottle \
    DATASET.name=MVTec \
    MODEL.name=PatchCore \
    MODEL.sampler=greedy_coreset \
    MODEL.sampler_ratio=0.1 \
    DEFAULT.savedir=./results/MVTec/PatchCore/greedy0.1

📊 Results

Results are saved in the specified savedir:

results/
├── MVTec/
│   ├── HierarchicalPatchCore/
│   │   └── greedy0.1-layer4/
│   │       ├── config.yaml          # Experiment configuration
│   │       ├── test_results.json    # Per-class & overall metrics
│   │       ├── test_binary_results.json  # Binary classification results
│   │       ├── eval_args.pkl        # Evaluation arguments
│   │       └── infer_time.json      # Inference time statistics
│   └── PatchCore/
│       └── greedy0.1/
└── ...

🔧 Project Structure

HierCore/
├── main.py                 # Main training/evaluation script
├── configs.yaml            # Default configuration
├── models/
│   ├── factory.py          # Model factory
│   ├── patchcore.py        # PatchCore implementation
│   └── hierarchical_patchcore.py  # HierCore implementation
├── backbones.py            # Backbone networks (WideResNet50, etc.)
├── common.py               # Common utilities (Feature aggregator, FAISS NN)
├── dataset.py              # Dataset loaders
├── metrics.py              # Evaluation metrics
├── sampler.py              # Coreset sampling methods
├── router.py               # Routing module for inference
├── runs/                   # Shell scripts for experiments
│   ├── run.sh              # Run all experiments
│   ├── run_mvtec.sh        # MVTec AD experiments
│   ├── run_visa.sh         # VisA experiments
│   ├── run_btad.sh         # BTAD experiments
│   └── run_mpdd.sh         # MPDD experiments
├── notebook/               # Analysis notebooks
├── results/                # Experiment results
└── docker/                 # Docker configuration

📝 Citation

If you find this work useful, please cite our paper:

@article{hiercore2026,
  title={Multi-class Image Anomaly Detection for Practical Applications: Requirements and Robust Solutions},
  author={Jaehyuk Heo, Pilsung Kang},
  journal={Neurocomputing},
  year={2026},
  publisher={Elsevier}
}

🙏 Acknowledgements

This work builds upon the following excellent projects:

• PatchCore - Original PatchCore implementation
• FINCH Clustering - First Integer Neighbor Clustering Hierarchy
• timm - PyTorch Image Models

📄 License

This project is released under the MIT License.

📧 Contact

For questions or issues, please open an issue on GitHub or contact the authors.