Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

In Pytorch F.nll_loss() Expected object of type torch.LongTensor but found type torch.FloatTensor for argument #2 'target'

Why does this error occur.

I am trying to write a custom loss function, that finally has a negative log likelihood.

As per my understanding the NLL is calculated between two probability values?

>>> loss = F.nll_loss(sigm, trg_, ignore_index=250, weight=None, size_average=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home//lib/python3.5/site-packages/torch/nn/functional.py", line 1332, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
RuntimeError: Expected object of type torch.LongTensor but found type torch.FloatTensor for argument #2 'target'
The inputs here being the following: 
>>> sigm.size()
torch.Size([151414, 80])
tensor([[ 0.3283,  0.6472,  0.8278,  ...,  0.6756,  0.2168,  0.5659],
        [ 0.6603,  0.5957,  0.8375,  ...,  0.2274,  0.4523,  0.4665],
        [ 0.5262,  0.4223,  0.5009,  ...,  0.5734,  0.3151,  0.2076],
        [ 0.4083,  0.2479,  0.5996,  ...,  0.8355,  0.6681,  0.7900],
        [ 0.6373,  0.3771,  0.6568,  ...,  0.4356,  0.8143,  0.4704],
        [ 0.5888,  0.4365,  0.8587,  ...,  0.2233,  0.8264,  0.5411]])
And my target tensor is:     
>>> trg_.size()
torch.Size([151414])
tensor([-7.4693e-01,  3.5152e+00,  2.9679e-02,  ...,  1.6316e-01,
         3.6594e+00,  1.3366e-01])
If I convert this to long I loose all data:      
>>> sigm.long()
tensor([[ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0],
        [ 0,  0,  0,  ...,  0,  0,  0]])
>>> trg_.long()
tensor([ 0,  3,  0,  ...,  0,  3,  0])
If I convert the raw values of target tensor to sigmoid too: 
>>> F.sigmoid(trg_)
tensor([ 0.3215,  0.9711,  0.5074,  ...,  0.5407,  0.9749,  0.5334])
>>> loss = F.nll_loss(sigm, F.sigmoid(trg_), ignore_index=250, weight=None, size_average=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/lib/python3.5/site-packages/torch/nn/functional.py", line 1332, in nll_loss
    return torch._C._nn.nll_loss(input, target, weight, size_average, ignore_index, reduce)
RuntimeError: Expected object of type torch.LongTensor but found type torch.FloatTensor for argument #2 'target'
This does calculate the loss happily, but it is again just make belief as I have lost data in the long conversion: 
>>> loss = F.nll_loss(sigm, F.sigmoid(trg_).long(), ignore_index=250, weight=None, size_average=True)
tensor(-0.5010)
>>> F.sigmoid(trg_).long()
tensor([ 0,  0,  0,  ...,  0,  0,  0])
"As per my understanding, the NLL is calculated between two probability values?"
No, NLL is not calculated between two probability values. As per the pytorch docs 
(See shape section), It is usually used to implement cross entropy loss. It takes input which is expected to be log-probability and is of size (N, C) when N is data size and C is the number of classes. Target is a long tensor of size (N,) which tells the true class of the sample.
Since in your case, target for sure is not the true class, you might have to implement your own version of loss and you may not be able to use NLLLoss. If you add more details about what loss you want to code up, I can help/explain more on how to do that (if possible by using existing function in torch).
I'm just going to leave the runnable minimal commented code here that would allow you to see dimensions at each step and understand how this (or other) losses work:
import torch
import torch.nn as nn
m = nn.LogSoftmax()
loss = nn.NLLLoss()
# input is of size N x C = 3 x 5
# this is FloatTensor containing probability for 
# each item in batch for each class
input = torch.randn(3, 5)
# target is LongTensor for index of true class for each item in batch
# each element in target has to have 0 <= value < C
target = torch.tensor([1, 0, 4])
# output is tensor of 0 dimension, i.e., scaler wrapped in tensor
output = loss(m(input), target)
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.