allow Load offline, expose local path

[Borda]

Resolves #34. propagate exposing the local path to the contractor...

[Borda]

@jamt9000 it is used/tested in https://www.kaggle.com/jirkaborovec/score-toxic-comments-with-trained-detoxify

[laurahanu]

Thanks for the PR, sounds like this might be useful to have!

Since the things that are being downloaded by HF are only tokenizer related (e.g. tokenizer.json, tokenizer_config.json, vocab.txt, config.json) we should probably rename this to tokenizer_path and add it to the docstring too.

[Borda]

@laurahanu all resolved as you requested 🐰

[laurahanu]

Great, thanks! Although, would still be good to change pretrained_model_path to transformers_offline_files_path or required_offline_files_path or something that suggests it doesn't need the model weights

[Borda]

Great, thanks! Although, would still be good to change pretrained_model_path to transformers_offline_files_path or required_offline_files_path or something that suggests it doesn't need the model weights

not sure if I understand... to be able to run it offline you need to download the same files existing in the hub, but you wanted to rename the argument differently? if you really wish you you can do this edit in PR but I am not much happy about raising this confusion for user :/

[laurahanu]

Yes, since the files needed are only the config and tokenizer files, it seems more confusing if the variable is named pretrained_model_path when it is not intended as a path to the model weights. Especially when there is another argument, checkpoint, that is intended as that.

Also, would be good to add the new variable to the main class docstring too.

[Borda]

@laurahanu done

[laurahanu]

Great, thank you!

One last thing, would be great if we could mention in the docstring that this is needed for offline model loading, something like:

huggingface_config_path: path to HF config and tokenizer files needed for offline model loading

[Borda]

@laurahanu done

[jamt9000]

Thank you for your contributions and happy new year!

(Windows failure should be fixed by other pr)

[Borda]

(Windows failure should be fixed by other pr)

it is fixed over here #40

[Borda]

@jamt9000 @laurahanu could we merge it? 🐰