Pedestrian retrieval with various types of queries, e.g. a set of attributes or a portrait photo, has a great application potential in large-scale intelligent surveillance systems. This paper presents a Richly Annotated Pedestrian (RAP) dataset v2.0 for the pedestrian retrieval application, which is collected from uncontrolled multi-camera surveillance scenarios. RAP dataset v2.0 is an extended version of RAP dataset v1.0. RAP dataset v1.0 only contains 41585 attribute annotated pedestrian images. As an extension vision, RAP dataset v2.0 adds identity annotations for a part of v1.0 and collects more attribute annotated pedestrian images as well. In total, RAP dataset v2.0 has 84,928 attribute annotated pedestrian samples, and 26,638 of them are also identity annotated. The same as v1.0, each of pedestrian images in v2.0 is also annotated with viewpoints, occlusions, and body parts information, besides of 72 attributes. Based on this dataset, quantitative analysis are made by an amount of state-of-the-art algorithms on three tasks, i.e., pedestrian attribute recognition, attribute-based person retrieval and image-based person retrieval (person re-identification), to build a high-quality person retrieval benchmark. We hope the RAP dataset v2.0 can promote the research of person retrieval in real scenarios.

RAP dataset v2.0 aims to promote the research on pedestrian retrieval with various of queries, including pedestrian attributes and person images. The overall types of annotations are shown in Table 1.

Table 1: Overall annotations in the RAP dataset v2.0

Class		Attribute
Spatial-Temporal		Time, sceneID, image position, bounding box of body/head-shoulder/upper-body/lower-body/accessories.
Whole		Gender, age, body shape, role.
Accessories		Backpack, single shoulder bag, handbag, plastic bag, paper bag etc.
Posture,Actions		Viewpoints, telephoning, gathering, talking, pushing, carrying etc.
Occlusions		Occluded parts, occlusion types.
Parts	head	Hair style, hair color, hat, glasses.
	upper	Clothes style, clothes color.
	Lower	Clothes style, clothes color, footware style, footware color.

Pedestrian Attribute

The RAP dataset v2.0 contains 69 binary attributes and 3 multi-class attributes, such as gender, backpack, and cloth types. Besides common pedestrian attributes, some attributes are firstly annotated in RAP dataset, such as person actions. Some samples are shown in Figure 1.

Figure 1: Samples with action attribute annotations.

Besides 72 attributes, pedestrian orientations, occlusion patterns, and coarse body parts are also annotated, which are useful for pedestrian related applications. Locations of body parts and accessories are annotated as well. Samples with body part annotations are shown in Figure 2. The green box annotates the location of accessories.

Figure 2: Samples with body part annotations.

The overall comparison with existing pedestrian attribute datasets is shown in Table 2.

Table 2: Comparison with existing pedestrian attribute datasets.

Pedestrian Identity

The RAP dataset v2.0 contains 2,589 person identities. Existing person ReID datasets usually only consider a short time period under the assumption that cloth appearances of the same persons are unchanged. Differently, the RAP dataset v2.0 is collected during a long time and there are 598 person identities that appear more than one day. The cloth appearances of these identities are not exactly the same during different days. Samples with cross-day person identity annotations are shown in Figure 3. Based on this dataset, researchers can develop more efficient algorithms for long-term person retrieval.

Figure 3: Samples with cross-day person identity annotations.

The overall comparison with existing person ReID datasets is shown in Table 3.

Table 3: Comparison with existing person ReID datasets.