Trying to convert torch.multinomial to PTE model

[vikasbalaga]

🐛 Describe the bug

Hi,

I have started working with Executorch and I am trying to convert sample models with different NN operators and converting them to .pte models.

I am currently trying to convert "torch.multinomial()" function to a .pte model using following python script :

import torch
from torch.export import export
from executorch.exir import to_edge

# Start with a PyTorch model that adds two input tensors (matrices)
class multinomial(torch.nn.Module):
  def __init__(self, num_samples=1, replacement=False):
      super().__init__()
      self.num_samples = num_samples
      self.replacement = replacement

  def forward(self, probs: torch.Tensor):
      """
      Args:
          probs (Tensor): A 1D or 2D tensor of probabilities. Each row should sum to 1 if 2D.

      Returns:
          Tensor: Sampled indices from the multinomial distribution.
      """
      if probs.dim() == 1:
          return torch.multinomial(probs, self.num_samples, self.replacement)
      elif probs.dim() == 2:
          return torch.stack([
              torch.multinomial(row, self.num_samples, self.replacement)
              for row in probs
          ])
      else:
          raise ValueError("Input must be 1D or 2D tensor of probabilities.")

# 1. torch.export: Defines the program with the ATen operator set.
aten_dialect = export(multinomial(), (torch.ones(1),))

# 2. to_edge: Make optimizations for Edge devices
edge_program = to_edge(aten_dialect)

# 3. to_executorch: Convert the graph to an ExecuTorch program
executorch_program = edge_program.to_executorch()

# 4. Save the compiled .pte program
with open("multinomial.pte", "wb") as file:
    file.write(executorch_program.buffer)

But I am facing the below errors :

Operator torch._ops.aten.multinomial.default is not in Core ATen opset (https://pytorch.org/docs/stable/torch.compiler_ir.html#core-aten-ir)."
There are a few things to try:
1. You can proceed with `to_edge(compile_config=EdgeCompileConfig(_core_aten_ops_exception_list=[torch.ops.aten.multinomial.default]))`.
   Please make sure that the backend(s) you are planning to lower to is able to handle aten.multinomial.default, or you have a corresponding kernel linked to your runtime.

2. Sometimes inference and training gives slightly different op set. Try adding `with torch.no_grad():` context manager if you are export for inference only.

3. If the error persists after 2, this is likely caused by torch.export() + core ATen decomposition producing unexpected operators for your model.
   If you believe this operator should be included into core ATen opset, please create an issue in https://github.com/pytorch/pytorch/issues and add `module: core aten` tag.

Now, I know that multinomial() is not in Core ATen Ops list but so is Silu() and in the latter case, it is decomposed into sigmoid() and mul().

So, shouldn't it be the same case for multinomial also?

Versions

Collecting environment information...
PyTorch version: 2.6.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: version 4.0.0
Libc version: glibc-2.31

Python version: 3.10.0 (default, Mar 3 2022, 09:58:08) [GCC 7.5.0] (64-bit runtime)
Python platform: Linux-5.15.133.1-microsoft-standard-WSL2-x86_64-with-glibc2.31
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 39 bits physical, 48 bits virtual
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 140
Model name: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
Stepping: 1
CPU MHz: 2419.200
BogoMIPS: 4838.40
Virtualization: VT-x
Hypervisor vendor: Microsoft
Virtualization type: full
L1d cache: 192 KiB
L1i cache: 128 KiB
L2 cache: 5 MiB
L3 cache: 8 MiB
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect md_clear flush_l1d arch_capabilities

Versions of relevant libraries:
[pip3] executorch==0.5.0
[pip3] numpy==2.0.0
[pip3] torch==2.6.0+cpu
[pip3] torchao==0.8.0+gitebc43034
[pip3] torchaudio==2.6.0
[pip3] torchsr==1.0.4
[pip3] torchvision==0.21.0+cpu
[conda] executorch 0.5.0 pypi_0 pypi
[conda] numpy 2.0.0 pypi_0 pypi
[conda] torch 2.6.0+cpu pypi_0 pypi
[conda] torchao 0.8.0+gitebc43034 pypi_0 pypi
[conda] torchaudio 2.6.0 pypi_0 pypi
[conda] torchsr 1.0.4 pypi_0 pypi
[conda] torchvision 0.21.0+cpu pypi_0 pypi

[vikasbalaga]

Update:

2. Sometimes inference and training gives slightly different op set. Try adding with torch.no_grad(): context manager if you are export for inference only.

I have tried this, but the issue still persists