Images of the Russian Empire:

CS194-26 Proj 1 (Project Web Page) : Colorizing the Prokudin-Gorskii photo collection;

Project Overview

In this project, we were given a set of images taken by Sergei Mikhailovich Prokudin-Gorskii in the early 1900s. Ahead of his time, he took pictures of countless landmarks using three expores of every scene using a red, blue, and green filter. While there was no way to integrate the three photographs into a singular color image at the time, our project today is to align the three images taken, layer them on top of each other, and then use the in-built Python libraries to display the images.

My approach

My approach to this project was to develop a script on Jupyter notebook that could successfully process a single image, and use the algorithm and techniques to develop abstractions that could generalize to all of the images in the data set. I used the boilerplate code provided and noticed all I needed to implement was the `align` function. I then shifted the first image 15 pixels vertically and horizontally and then overlayed the two images. For each of the displacements, I calculate a score that corresponded to the difference of the images (lower the score the closer the two images are aligned) and then use the displacement with the minimum score. This score function are based on those provided: SSD (sum of squared differences) and NCC (normalizd cross correlation).

Challenges

Once I completed the basic implementation, it worked well for the smaller images, but I underestimated how long it would actually take for the larger images. Then I proceeded to implement the pyramiding function. I implemented pyramid aligning with an iterative approach as follows:

First I down sampled the two images by 50% until they were less than 300 pixels tall. I have a preset max_dispacement used to find the best vertical and horizontal dislacements for this blurriest image. Then I use the images one scale above, align the two images based on the displacements I found for the previous down sampled images, and then fine tune the adjustments. I continue this process until I've fine tuned the alignment for the original images.

Here's how it works:

Most downsampled (300px x 300px) ~ adjust by +- MAX_DISP (ex. 8).
Then, one level higher (600 x 600) ~ adjust by +- MAX_DISP / 2 (ex. 4)
Then, one level higher (1200 x 1200) ~ adjust by +- MAX_DISP / 4 (ex. 2)
.. until we fine tune the original image by 1 pixel

Another challenge I had was when I was calculating the score between two images, I originally used the whole images. But the black borders from the image, and the colorful borders from shifting the image added unncessary noise and made some images seem incorrectly more similar or different than they actually were. To solve this issue, I added another parameter called frame_size; I only compared a frame within each of the images a small subsction of the center of each image.

Results

Small Images

Large Images

Bonus Pictures

Bells and Whistles

Edge Detection

Notice how the Green and Blue images were already aligned perfectly originally. The Red and Blue filtered images are the problem. Once we take an image with everything filtered except its edges, it works. Let's take a look at what the edges look like

The Canny edge detector uses a variance parameter which you can use to specify how sensitive you want the edge detector to be. If you set a high standard deviation it returns fewer pixels indicated to be part of an edge.

Let's look at the difference in image quality with a sigma of 5 instead of 1

The differences are minimal with this change.

Histogram Equalization

Note this was implemented using the skimage libraries provided

Images of the Russian Empire:

Project Overview

My approach

Challenges

Results

Small Images

Cathedral

Monastery

Settlers

Nativity

Large Images

Harvesters

Emir

Icon

Lady

Self Portrait

Three Generations

Train

Turkmen

Village

Bonus Pictures

Chapel where Ivan the Terrible's Son was Born!

Floodgate and guardhouse of the M.P.S.

Monument

Bells and Whistles

Edge Detection

Original Aligned Emir (no edge detection)

Aligned Emir (with Canny edge detection)

Emir filtered Red with Canny Edge Detector Applied

Emir filtered Blue with Canny Edge Detector Applied

Emir filtered Red with Canny Edge Detector Applied

Emir filtered Blue with Canny Edge Detector Applied

Original Aligned Emir (no edge detection)

Aligned Emir (with Canny edge detection - Sigma - 5)

Histogram Equalization

Original Aligned Lady (no Histogram Equalization)

Aligned Lady (with histogram equalization)