Nvidia Shown Training Neural Networks on a Small Data Set

At Nvidia Research presented an artificial intelligence model that uses a limited set of data to train generative neural networks and is capable of emulating famous artists.

The new training technique was applied to the StyleGAN2 model. She was trained on less than 1,500 images from the Metropolitan Museum of Art. Using DGX systems to accelerate learning, the model created its own pictures.

A technique called Adaptive Discriminator Extension, or ADA, reduces the number of training images by a factor of 10 to 20 while providing good results.

Like most neural networks, GANs followed a basic principle: the more training data, the better the model. Typically, training them requires 50 to 100 thousand images. But in many cases, researchers simply do not have such a base at their disposal. Artificially augmenting the training dataset with distorted copies of existing images causes the GAN to simulate these distortions rather than creating believable synthetic images.

The ADA method adaptively applies data padding. This means that the amount of augmented data is adjusted at different stages of the learning process to avoid overfitting the GAN.

This method can be applied not only in art but also in healthcare, for example, by creating histological images of cancer to train artificial intelligence models.
“This means that people can use the GAN to solve problems when huge amounts of data take too much time or effort to explore,” said David Lübcke, vice president of graphics research.

Researchers have already pay attention to the fact that it is more difficult to reveal the potential of AI in the medical field. As a rule, there is not enough training data, and there is also a risk of disclosing confidential information if work is carried out, for example, with medical records. Nvidia says the new training method will preserve privacy.