Vintage train reaches a new station

AI restores 1896 film

Technological advances are always thought to be a leap forward – cutting off all ties with the past. However, technology is increasingly being used to preserve or even restore past artefacts to their former glory. The latest in this regard is the use of Artificial Intelligence in restoring vintage movies. Developer Denis Shiryaev has recently upgraded an antique black-and-white film to 4K format using a cocktail of AI-powered tools – and the result is encouraging.

In January 1896, the Lumière brothers – Auguste and Louis – screened a 50 second film clip in Paris, titled L’arrivée d’un train en gare de La Ciotat (The Arrival of a Train at La Ciotat Station). One of the earliest films by the two pioneering French moviemakers, it showed just what its title describes – a steam locomotive pulling into a station at the French coastal town of La Ciotat.

Vintage train reaches a new station:
Comparative images

Original footage shown in 1896

4K version upscaled by Denis Shiryaevin 2020

Silent, grainy and in black-and-white, the visual was still novel enough for the viewers unaccustomed to moving pictures to jump off their seats and rush in search of safety – thinking that the train was about to leap out of the screen! This is the film that Shiryaev has restored through AI.

The method used is innovatively simple and the basic approach has been around for over two decades. As TV screens turned more and more upscale with ever increasing display resolutions, the number of pixels on screen skyrocketed as well. Check out the following comparison:

	Display resolution	Pixels on-screen
Old standard definition TV sets	720 x 480	345,600
Advanced high definition sets	1920 × 1080	2,073,600
State-of-the-art 4K TV sets	3840 x 2160	8,294,400

This means, the more updated TV set you have, additional pixels will have to be filled in on screen to display the same image. The math is simple. The image on your old standard definition TV occupies 345,600 pixels on screen, but the moment it is beamed on a 4K screen with 8,294,400 pixels, an extra 7,948,800 pixels must be populated with the same content. This is done by a process called interpolation, which decides what to fill in the extra pixels with by evaluating the colours of the already-populated pixels around. This can be done through any of the following three methods:

• Nearest neighbour method: Populates a blank pixel with the same colour as its nearest filled-in pixel; the resulting image is jagged.
• Bilinear interpolation: Analyses two adjacent filled-in pixels and creates a shade in-between; the results are sharper than the previous method.
• Bicubic interpolation: Here, analysis is made based on 16 nearest pixels to come up with the right colour for the blank pixel; this is much precise but still a bit blurred.

For optimum results, bilinear and bicubic methods are used together to make up for the inadequacies of each other.

Obviously, a lot of calculation is required to get the right pixel colour – and this goes on for every extra pixel to be populated. Shiryaev has simply entrusted this calculation job to AI tools. He used two existing enhancement programs, DAIN and Gigapixel AI for this purpose.

DAIN analyses existing video clips using deep convolutional neural networks, and then generates filler frames to be inserted between the keyframes of the clip – thus creating a 60 frames per second output.

Gigapixel AI, designed by Topaz Labs, uses its own interpolation algorithm to identify details and structures in an image and provides finishing touches to create a sharp output even after 600% enlargements.

Combining both tools, Shiryaev converted the original 1896 film into a 4K 60FPS clip. The results are available on YouTube. Definitely not crystal-clear yet, but Shiryaev’s work opens up an immense possibility where vintage celluloid can be given a fresh lease of life and preserved for posterity with ready help from AI.