The Chars74K dataset Character Recognition in Natural Images

[ jump to download ]

Character recognition is a classic pattern recognition problem for which researchers have worked since the early days of computer vision. With today's omnipresence of cameras, the applications of automatic character recognition are broader than ever. For Latin script, this is largely considered a solved problem in constrained situations, such as images of scanned documents containing common character fonts and uniform background. However, images obtained with popular cameras and hand held devices still pose a formidable challenge for character recognition. The challenging aspects of this problem are evident in this dataset.

In this dataset, symbols used in both English and Kannada are available.

In the English language, Latin script (excluding accents) and Hindu-Arabic numerals are used. For simplicity we call this the "English" characters set. Our dataset consists of:

• 64 classes (0-9, A-Z, a-z)
• 7705 characters obtained from natural images
• 3410 hand drawn characters using a tablet PC
• 62992 synthesised characters from computer fonts

The compound symbols of Kannada were treated as individual classes, meaning that a combination of a consonant and a vowel leads to a third class in our dataset. Clearly this is not the ideal representation for this type of script, as it leads to a very large number of classes. However, we decided to use this representation for our baseline evaluations present in [deCampos et al] as a way to evaluate a generic recognition method for this problem.

Reference

The following paper gives further descriptions of this dataset and baseline evaluations using a bag-of-visual-words approach with several feature extraction methods and their combination using multiple kernel learning:

T. E. de Campos, B. R. Babu and M. Varma. Character recognition in natural images. In Proceedings of the International Conference on Computer Vision Theory and Applications (VISAPP), Lisbon, Portugal, February 2009.
Bibtex | Abstract | PDF

Download

Disclaimer: by downloading and using the datasets below (or part of them) you agree to acknowledge their source and cite the above paper in related publications. We will be grateful if you contact us to let us know about the usage of the our datasets.

• Images of individual characters: the files below contain directory trees of each dataset of individual characters. In these trees, there is one directory per class of character. Each character sample appear in an individual PNG image. There's a large variation is scale, as we kept the original resolution of the characters as they appear in the original images.
• English, 62 classes (0-9, A-Z, a-z)
• EnglishImg.tgz (127.9 MB) [sample characters]: segmented characters from natural scenes. For each character, a binary segmentation mask file is also provided.
• EnglishHnd.tgz (13.0 MB) [sample characters]: hand-drawn characters. 55 samples per class. The pen stroke trajectories are also provided, so this dataset can also be used to evaluate on-line handwritten character recognition methods.
• EnglishFnt.tgz (51.1 MB): characters from computer fonts with 4 variations (combinations of italic, bold and normal).
• Kannada (657+ classes)
• KannadaImg.tgz (105.8 MB) [sample characters]: segmented characters from natural scenes.
• KannadaHnd.tgz (125.8 MB) [sample characters]: hand-drawn characters.
• Lists.tgz (7.3 MB): lists of files used for training and testing in our experiments. Please use these splits in order to make fair comparisons with the results published in the paper above.
• FullImagesAndAnnotations_Frontal.tgz (739.7 MB) [sample images]: original images and TXT files which give the coordinates, bounding polygons and labels of characters that appear in each image. Many whole words have also been annotated. These annotations may be used to evaluate character/word detection methods, but not all the words that appear in the images have been annotated.
• Maps.tgz (1.3 MB): inverse maps, points each sample character in EnglishImg.tgz (or KannadaImg.tgz) to the original full image in FullImagesAndAnnotations_frontal.tgz.

Sample characters

(In addition to the ones shown following the links above)