Evaluation dataset

The evaluation dataset will be composed of 9 books. It will be composed of a set of training images and a set of test images. The training dataset will contain a reduced number of book pages, along with their ground truth in the TXT format. The training images are representative of different contents and layouts of the book pages. On the other side, the test dataset will be composed of images representing the remainder book pages.

Only the ground truths of few pages of the 9 selected books will be provided to constitute the training dataset of the evaluation dataset.

The participants are free to use the evaluation dataset for training, testing or any other purpose related to the HBA competition.

The nine selected books to form the evaluation dataset are: Book 1, Book 2, Book 3 and Book 5 as manuscript books, and Book 7, Book 8, Book 9, Book 10 and Book 11 as printed ones.

The training dataset of each book of the evaluation dataset is structured as follows.

Book Id. Number of training images
Book 1 22
Book 2 42
Book 3 56
Book 5 27
Book 7 24
Book 8 45
Book 9 20
Book 10 26
Book 11 32

All evaluation dataset files are available from this link. The evaluation dataset is only available for registered participants. A login and a password will be sent to each registered participant of the HBA competition.

Each book of the evaluation dataset is composed of 3 folders namely, “images”, “train” and “test”.

  • “images”: It contains all TIFF images of a book.
  • “train”: It is composed of a number of TXT files to form the training dataset. The training dataset is representative of different contents and layouts of the analyzed book pages. Each line of a TXT file is composed of the following three values: the coordinates of the selected foreground pixel and its corresponding label class representing the content type in the analyzed book. The label value varies between 1 and 6. If the label value is equal to 1, the content class represents a graphical content else it corresponds to a textual content.
  • “test”: It contains the remainder of book TXT files by reference to the training dataset in order to form the test dataset. Each line of a TXT file is composed of only the coordinates of the selected foreground pixel. The participants should fill out these files with the class predicted label for each foreground pixel.