The accuracy when training with out groud_truth image is quite low

[anewworl]

So this is the custom.yaml file of mine in this test i remove the ground truth file and run task classification instead of segmentation. The aim of this is testing the accuracy with out the ground truth image (mask). The result is not reaching my expectation because after running with 10 epochs is likely to miss a lot abnormal and the percentage also not high. Below is my custom.yaml:

dataset:
  name: mvtec
  format: folder
  path: ./datasets/MVTec
  category: bottle
  normal_dir: bottle/train/good # name of the folder containing normal images.
  abnormal_dir: bottle/train/bad # name of the folder containing abnormal images.
  task: classification # classification or segmentation
  normal_test_dir: null # optional
  extensions: null
  mask_dir: null #optional 
  train_batch_size: 32
  eval_batch_size: 32
  num_workers: 8
  image_size: 256 # dimensions to which images are resized (mandatory)
  center_crop: null # dimensions to which images are center-cropped after resizing (optional)
  normalization: imagenet # data distribution to which the images will be normalized: [none, imagenet]
  transform_config:
    train: null
    eval: null
  test_split_mode: from_dir # options: [from_dir, synthetic]
  test_split_ratio: 0.2 # fraction of train images held out testing (usage depends on test_split_mode)
  val_split_mode: same_as_test # options: [same_as_test, from_test, synthetic]
  val_split_ratio: 0.5 # fraction of train/test images held out for validation (usage depends on val_split_mode)
  tiling:
    apply: false
    tile_size: null
    stride: null
    remove_border_count: 0
    use_random_tiling: False
    random_tile_count: 16

model:
  name: padim
  backbone: resnet18
  pre_trained: true
  layers:
    - layer1
    - layer2
    - layer3
  normalization_method: min_max # options: [none, min_max, cdf]

metrics:
  image:
    - F1Score
    - AUROC
  pixel:
    - F1Score
    - AUROC
  threshold:
    method: adaptive #options: [adaptive, manual]
    manual_image: null
    manual_pixel: null

visualization:
  show_images: False # show images on the screen
  save_images: True # save images to the file system
  log_images: True # log images to the available loggers (if any)
  image_save_path: null # path to which images will be saved
  mode: full # options: ["full", "simple"]

project:
  seed: 42
  path: ./results

logging:
  logger: [] # options: [comet, tensorboard, wandb, csv] or combinations.
  log_graph: false # Logs the model graph to respective logger.

optimization:
  export_mode: null # options: torch, onnx, openvino

# PL Trainer Args. Don't add extra parameter here.
trainer:
  enable_checkpointing: true
  default_root_dir: null
  gradient_clip_val: 0
  gradient_clip_algorithm: norm
  num_nodes: 1
  devices: 1
  enable_progress_bar: true
  overfit_batches: 0.0
  track_grad_norm: -1
  check_val_every_n_epoch: 1 # Don't validate before extracting features.
  fast_dev_run: false
  accumulate_grad_batches: 1
  max_epochs: 10
  min_epochs: null
  max_steps: -1
  min_steps: null
  max_time: null
  limit_train_batches: 1.0
  limit_val_batches: 1.0
  limit_test_batches: 1.0
  limit_predict_batches: 1.0
  val_check_interval: 1.0 # Don't validate before extracting features.
  log_every_n_steps: 50
  accelerator: auto # <"cpu", "gpu", "tpu", "ipu", "hpu", "auto">
  strategy: null
  sync_batchnorm: false
  precision: 32
  enable_model_summary: true
  num_sanity_val_steps: 0
  profiler: null
  benchmark: false
  deterministic: false
  reload_dataloaders_every_n_epochs: 0
  auto_lr_find: false
  replace_sampler_ddp: true
  detect_anomaly: false
  auto_scale_batch_size: false
  plugins: null
  move_metrics_to_cpu: false
  multiple_trainloader_mode: max_size_cycle

here is some result of the training above:

This is only sample which reah my requirments:

I use the MVTEC dataset, can anyone pls explain about my case is this normal or i have done anything wrong. tks a lot

[abc-125]

The ground truth masks are not used for training, only for testing. So, if you do not use them, the results should stay the same.
You have max_epochs: 10 in your config, but PaDiM is being "trained" for one epoch only, so it is better to change it to max_epochs: 1. To achieve more satisfying results, you can try different models, such as PatchCore. You can also look at the sample results to know what to expect. I hope this helps!

[aileenfun]

@anewworl
hi, have you find the solution? I meet the same problem here.