Posts

In the previous post I explained that, after a certain point, improving efficiency of the data loaders or increasing the number of data loaders have a marginal impact on the training speed of deep learning.

Have you wondered why some of your training scripts halt every n batches where n is the number of loader processes? This likely means your pipeline is bottlenecked by data loading time, as shown in the following animation:

These are some of the most frequent mistakes students made in the lab when I worked as a tutor for ICS 46, Data Structure Implementation and Analysis. Anyone who is taking the class could benefit by reviewing these examples and trying to figure out what is wrong with each of them.