Issue
Hi I am trying to use tensor board for the model given at - https://github.com/siddharthvaria/WordPair-CNN/blob/master/src/pytorch_impl/cnn_pdtb_arg_multiclass_jl.py
However I get an error at Line 135 : x_max_pools = [F.max_pool1d(xi, xi.size(2)).squeeze(2) for xi in x_convs]
As
max_pool1d(): argument ‘kernel_size’ must be tuple of ints. Is there any solution to resolve this?
There is a discussion about this in pytorch forum - https://discuss.pytorch.org/t/typeerror-avg-pool2d-argument-kernel-size-must-be-tuple-of-ints-not-proxy/108482
Solution
The line that raises error should be line 135:
# At this point x_convs is [(batch_size, nfmaps_arg, seq_len_new), ...]*len(fsz_arg)
x_max_pools = [F.max_pool1d(xi, xi.size(2)).squeeze(2) for xi in x_convs] # [(batch_size, nfmaps_arg), ...]*len(fsz_arg)
Actually max_pool1d
reduces the size of the dimension n°2 that represents the sequence (after batch and channel dimension), see the documentation. Plus, according to the discuss you point out, the kernel_size
argument now requires a concrete value at execution and cannot be inferred dynamically from inputs.
Fortunately your case is particularly simple because the kernel size is the entire sequence length. In other words, your pooling is actually a global max pooling that reduces the size of the sequence to only one scalar. This scalar is simply the maximum of all the values (check the shapes in comments!). So it is actually easy to implement a version without dynamic shapes. Just change line 135 by:
x_max_pools = [torch.max(xi, dim=2, keepdim=True)[0] for xi in x_convs] # [(batch_size, nfmaps_arg), ...]*len(fsz_arg)
Note:
- there is no direct implementation of
global_max_pooling
layer but as you can see it is just atorch.max
- Actually
torch.max
return a tuple(max, max_indicies)
so you hae to take only the first element with[0]
(see the doc) - gradients can propagate through
torch.max
like usualmax_pooling
layers (one source see here for instance) torch.max(x, dim=2).squeeze(2)
is changed totorch.max(x, dim=2, keepdim=True)
that is exactly the same thing but cleaner.
Answered By - Valentin Goldité
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.