While watching a video of Chinese calligraphy, I realized that unlike the English alphabet, there was no clear-cut order of generation of brushstrokes (or letters) that marked the completion of individual characters (words). In the world of computer generated graphics, it seemed disappointing that there existed no particularly elegant way for characters to appear on an image as the user wrote them. Hence, I thought it would be interesting to see if we could use computer vision to take a video of a person writing and remove all influences other than the ink that flowed onto the paper, making the characters “magically” appear.
Using OpenCV, my lab partner and I used thresholding and morphological operators to define and extract the areas in the video that corresponded to characters. The biggest challenge was to compensate for the writer’s hand, which often covered the written characters and the ink brush, which often matched the color intensity of the ink. By using connected component analysis to identify areas corresponding to skin tone, we were able to successful wrap around the obstructions of the hand region and minimize the amount of misidentification due to the brush .
The code runs relatively robustly in real time (half-speed), although the saved video is set to match the original’s frame rate.
*Github Repository for OpenCV code in Python available on request.