Av2SensorDatasetProcessor

class standard_e2e.caching.src_datasets.av2_sensor.Av2SensorDatasetProcessor(common_output_path, split, index_data_generator=None, adapters=None, context_aggregators=None)[source]

Bases: SourceDatasetProcessor

Processor for the Argoverse 2 sensor dataset.

A “frame” is a single lidar sweep timestamp; ring-camera images are matched by nearest timestamp per camera, and 3D box annotations are filtered for that exact sweep timestamp (AV2 annotates one cuboid per sweep, not per camera frame). One log = one segment.

Per-log state (calibration, ego-pose table, full annotations table, static map) is read once and reused across every sweep of the log; the cache is keyed by log_dir so workers that hop between logs only reload when they actually transition.

Parameters:
DATASET_NAME = 'av2_sensor'
property allowed_splits: list[str]

Return the list of allowed splits for the dataset.

property context_aggregators
property dataset_name: str

Return the name of the dataset.

property inner_path: str

Return the inner path relative to the common output path.

needs_attr(attr)

Whether at least one registered adapter reads this StandardFrameData field. Used by per-dataset processors to skip expensive modality builds (cameras, lidar, hd_map, detections, …) when no adapter would consume them. True when attr is in the consumed-attrs union, plus a hard-coded special case: the identifier / index fields are always treated as needed since they are required for the cache + index regardless of adapter chain.

Return type:

bool

Parameters:

attr (StandardFrameDataField)

property output_path: str

Return the output path for the processed dataset.

process_frame(raw_frame_data)
Return type:

tuple[TransformedFrameData, FrameIndexData]

Parameters:

raw_frame_data (Any)

process_frame_and_save_data(raw_frame_data)

Process a single frame of raw data, save the processed frame data to disk, and return the corresponding FrameIndexData.

Return type:

FrameIndexData

Parameters:

raw_frame_data (Any)

property specific_output_path: str

Return the specific output path for the dataset.

property split: str

Return the dataset split.