Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

I was trying to load the dataset .hdf5 format using the below code of shape (1000,) but i was having the error ValueError: Invalid location identifier (invalid location identifier) . The error is popping up when i try to load the dataset into the pytorch dataloader.

with h5py.File(dataset_path, 'r') as f:
    data = f['default']
    print(data.shape)
Ouput:
(1000,)
# Define the dataset
class MyDataset(Dataset):
    def __init__(self, dataset_path):
        super().__init__()
        with h5py.File(dataset_path, 'r') as f:
            self.data = f['default']
    def __len__(self):
        return len(self.data)
    def __getitem__(self, idx):
        return self.data[idx]
# Load the dataset
dataset_path = 'dataset.hdf5'
train_dataset = MyDataset(dataset_path)
train_loader = DataLoader(train_dataset, shuffle=True)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-80-c3c741b81eff> in <module>
     23 train_dataset = MyDataset(dataset_path)
---> 24 train_loader = DataLoader(train_dataset, shuffle=True)
6 frames
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
/usr/local/lib/python3.9/dist-packages/h5py/_hl/dataset.py in shape(self)
    473         with phil:
--> 474             shape = self.id.shape
    476         # If the file is read-only, cache the shape to speed-up future uses.
h5py/h5d.pyx in h5py.h5d.DatasetID.shape.__get__()
h5py/h5d.pyx in h5py.h5d.DatasetID.shape.__get__()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/_objects.pyx in h5py._objects.with_phil.wrapper()
h5py/h5d.pyx in h5py.h5d.DatasetID.get_space()
ValueError: Invalid dataset identifier (invalid dataset identifier)
and i am receiving the same error when i am doing this f.get("default")
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.