ClearML support for train_pytorch

[hanshupe007]

I saw that clearml is only supported in the train_tensorflow script, was there any reason why it's not implemented in train_pytorch?

[felixdittrich92]

Hi @hanshupe007 👋,
If I remember we had a problem with a memory leak.
But we have still an open ticket for it: #1458
CC @odulcy-mindee

[odulcy-mindee]

Hello ! Yes, a memory leak is observed when ClearML is used. Few PRs has been merged and IIRC one PR fixed an unclosed open(...) so maybe it has been fixed. I haven't tested for a while.

[hanshupe007]

Is there a version I can pull with ClearML support? because the current train_pytorch.py doesn't include any clearml code.

[felixT2K]

Hi @hanshupe007 👋,
Unfortunately no depends on @odulcy-mindee open ticket (#1458)

[pavelgein]

I found this issue while googling for known issues with Clearml, because we have encountered memory leak. A colleague of mine find the problem and opened the issue clearml/clearml#1556

ClearML team says, that the Task should be inited as soon as possible, maybe this will be relevant to you