Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

import torchtext.vocab as vocab
custom_embeddings = vocab.Vectors(name = 'custom_embeddings.txt')
TEXT.build_vocab(train_data, vectors = custom_embeddings)
Here is the code for the Encoder class:
class Encoder(nn.Module):
    def __init__(self, 
                 input_dim, 
                 hid_dim, 
                 n_layers, 
                 n_heads, 
                 pf_dim,
                 dropout, 
                 device,
                 max_length = 100):
        super().__init__()
        self.device = device
        self.tok_embedding = nn.Embedding(input_dim, hid_dim)
        # step added for custom embedding
        self.tok_embedding.weight.data.copy_(custom_embeddings)
        self.pos_embedding = nn.Embedding(max_length, hid_dim)
        self.layers = nn.ModuleList([EncoderLayer(hid_dim, 
                                                  n_heads, 
                                                  pf_dim,
                                                  dropout, 
                                                  device) 
                                     for _ in range(n_layers)])
        self.dropout = nn.Dropout(dropout)
        self.scale = torch.sqrt(torch.FloatTensor([hid_dim])).to(device)
    def forward(self, src, src_mask):
        #src = [batch size, src len]
        #src_mask = [batch size, src len]
        batch_size = src.shape[0]
        src_len = src.shape[1]
        pos = torch.arange(0, src_len).unsqueeze(0).repeat(batch_size, 1).to(self.device)
        #pos = [batch size, src len]
        src = self.dropout((self.tok_embedding(src) * self.scale) + self.pos_embedding(pos))
        #src = [batch size, src len, hid dim]
        for layer in self.layers:
            src = layer(src, src_mask)
        #src = [batch size, src len, hid dim]
        return src
Now when I am trying to create the Encoder object, I am getting error for the custom embedding I have used.
enc = Encoder(INPUT_DIM, 
              HID_DIM, 
              ENC_LAYERS, 
              ENC_HEADS, 
              ENC_PF_DIM, 
              ENC_DROPOUT, 
              device)
Error decscription:
TypeError                                 Traceback (most recent call last)
<ipython-input-72-06d3631c029b> in <module>()
     18               ENC_PF_DIM,
     19               ENC_DROPOUT,
---> 20               device)
     22 dec = Decoder(OUTPUT_DIM, 
<ipython-input-59-6c2f23451d01> in __init__(self, input_dim, hid_dim, n_layers, n_heads, pf_dim, dropout, device, max_length)
     17         # step added for custom embedding
---> 18         self.tok_embedding.weight.data.copy_(custom_embeddings)
     20         self.pos_embedding = nn.Embedding(max_length, hid_dim)
TypeError: copy_(): argument 'other' (position 1) must be Tensor, not Vectors
Could you please help me to fix this error?
Thanks in advance!
Solution:
custom_embeddings is a vector object so we can not .copy_ from it as it is not tensor.
using TEXT.vocab.vectors in place of custom_embeddings resolved the issue.
updated code:
# step added for custom embedding
self.tok_embedding.weight.data.copy_(TEXT.vocab.vectors)
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.