A Richly Annotated Pedestrian Dataset for Person Retrieval in Real Surveillance Scenarios

Dangwei Li, Zhang Zhang, Xiaotang Chen, Kaiqi Huang


Pedestrian retrieval with various types of queries, e.g. a set of attributes or a portrait photo, has a great application potential in large-scale intelligent surveillance systems. This paper presents a Richly Annotated Pedestrian (RAP) dataset v2.0 for the pedestrian retrieval application, which is collected from uncontrolled multi-camera surveillance scenarios. RAP dataset v2.0 is an extended version of RAP dataset v1.0. RAP dataset v1.0 only contains 41585 attribute annotated pedestrian images. As an extension vision, RAP dataset v2.0 adds identity annotations for a part of v1.0 and collects more attribute annotated pedestrian images as well. In total, RAP dataset v2.0 has 84,928 attribute annotated pedestrian samples, and 26,638 of them are also identity annotated. The same as v1.0, each of pedestrian images in v2.0 is also annotated with viewpoints, occlusions, and body parts information, besides of 72 attributes. Based on this dataset, quantitative analysis are made by an amount of state-of-the-art algorithms on three tasks, i.e., pedestrian attribute recognition, attribute-based person retrieval and image-based person retrieval (person re-identification), to build a high-quality person retrieval benchmark. We hope the RAP dataset v2.0 can promote the research of person retrieval in real scenarios.

RAP Dataset v2.0

RAP dataset v2.0 aims to promote the research on pedestrian retrieval with various of queries, including pedestrian attributes and person images. The overall types of annotations are shown in Table 1.

Table 1: Overall annotations in the RAP dataset v2.0
Class Attribute
Spatial-Temporal Time, sceneID, image position, bounding box of body/head-shoulder/upper-body/lower-body/accessories.
Whole Gender, age, body shape, role.
Accessories Backpack, single shoulder bag, handbag, plastic bag, paper bag etc.
Posture,Actions Viewpoints, telephoning, gathering, talking, pushing, carrying etc.
Occlusions Occluded parts, occlusion types.
Parts head Hair style, hair color, hat, glasses.
upper Clothes style, clothes color.
Lower Clothes style, clothes color, footware style, footware color.

  • Pedestrian Attribute

    The RAP dataset v2.0 contains 69 binary attributes and 3 multi-class attributes, such as gender, backpack, and cloth types. Besides common pedestrian attributes, some attributes are firstly annotated in RAP dataset, such as person actions. Some samples are shown in Figure 1.

    Figure 1: Samples with action attribute annotations.

    Besides 72 attributes, pedestrian orientations, occlusion patterns, and coarse body parts are also annotated, which are useful for pedestrian related applications. Locations of body parts and accessories are annotated as well. Samples with body part annotations are shown in Figure 2. The green box annotates the location of accessories.

    Figure 2: Samples with body part annotations.

    The overall comparison with existing pedestrian attribute datasets is shown in Table 2.

    Table 2: Comparison with existing pedestrian attribute datasets.

  • Pedestrian Identity

    The RAP dataset v2.0 contains 2,589 person identities. Existing person ReID datasets usually only consider a short time period under the assumption that cloth appearances of the same persons are unchanged. Differently, the RAP dataset v2.0 is collected during a long time and there are 598 person identities that appear more than one day. The cloth appearances of these identities are not exactly the same during different days. Samples with cross-day person identity annotations are shown in Figure 3. Based on this dataset, researchers can develop more efficient algorithms for long-term person retrieval.

    Figure 3: Samples with cross-day person identity annotations.

    The overall comparison with existing person ReID datasets is shown in Table 3.

    Table 3: Comparison with existing person ReID datasets.

  • Download

    If you want to use the RAP dataset for scientific research only, please fill the aggrement here .
    The related codes are released. Recently a strong baseline has also been released.


    Please kindly cite our work if it helps your research: