❓ Help: Could you please help on an issue while exporting models to ONNX?

[fatcat-z]

I've tried to run the model with en/de/es succssfully, and we want to export the models to ONNX file for running with onnxruntime.

Unfortunately, in Ubuntu + PyTorch 1.7.1 + Python 3.8.5, the torch.onnx.export() functions failed and threw an exception with errors below:

temporary: the only valid use of a module is looking up an attribute but found = prim::SetAttr[name="num_batches_tracked"](%4243, %4373)

The code looks like below:

    model, _, utils = torch.hub.load(github, model_name, language=language_name)
    model.eval()
    (read_batch, split_into_batches, _, prepare_model_input) = utils 

    # download a single file, any format compatible with TorchAudio (soundfile backend)
    torch.hub.download_url_to_file('https://opus-codec.org/static/examples/samples/speech_orig.wav',
                                    dst = 'speech_orig.wav', progress=True)
    test_files = glob('speech_orig.wav')
    batches = split_into_batches(test_files, batch_size=10)
    input_tensor = (prepare_model_input(read_batch(batches[0])), )
    if torch.cuda.is_available():
        model.to('cuda')
        if isinstance(input_tensor, tuple):
            new_tensor = [x.to('cuda') for x in input_tensor]
            input_tensor = tuple(new_tensor)
        else: 
            input_tensor = input_tensor.to('cuda')
    torch.onnx.export(model, input_tensor, 'my_model.onnx', verbose=False, opset_version=12)

I also noticed that you have already prepared some ONNX files for these models. So, is it possible that you can share with me how you export those files successfully? With any special scripts?

Thanks!

[snakers4]

Hi,

and we want to export the models to ONNX file for running with onnxruntime.

Is there anything wrong with the provided ONNX checkpoints?

https://github.com/snakers4/silero-models/blob/master/models.yml#L10
https://github.com/snakers4/silero-models/blob/master/models.yml#L35
https://github.com/snakers4/silero-models/blob/master/models.yml#L44

The code looks like below:

Not sure I can replicate this, but typically ONNX export requires enumerations of input / output axes and I have not tried doing this with GPU tensors

[fatcat-z]

Understood. Anyway, thanks for your detailed explanations!

[fatcat-z]

Yes, of course

Just another question: does your original module contain an operator hann_window? If so, may I know how you export the module into ONNX file as provided in the github?

The torch.onnx.export() complains that this operator(hann_window) was not supported yet when I tried to export the script module.

[snakers4]

ONNX export has an option to freeze fixed params

We do not use the hann window operator, we just take the window itself from librosa or scipy

[fatcat-z]

ONNX export has an option to freeze fixed params

We do not use the hann window operator, we just take the window itself from librosa or scipy

I tried to load a module via the code in PyTorch hub:
model, decoder, utils = torch.hub.load(repo_or_dir='snakers4/silero-models',
model='silero_stt',
language='en', # also available 'de', 'es'
device=device)

By checking the code of returned model, I can see it contains code like below:
_3 = torch.hann_window(self.n_fft, dtype=ops.prim.dtype(x), layout=None, device=ops.prim.device(x), pin_memory=None)

It turns out the module loaded in this way will call torch.hann_window() method. Does it mean the logic of this returned module is a little different than the original module used to export ONNX file by you or your team?

[snakers4]

We had to do some tweaks to fit the limited number of ONNX operators
ONNX does not have stft, for example