WaymoPerceptionDatasetProcessor¶

class standard_e2e.caching.src_datasets.waymo_perception.WaymoPerceptionDatasetProcessor(common_output_path, split, index_data_generator=None, adapters=None, context_aggregators=None)[source]¶

Bases: SourceDatasetProcessor

Processor for the Waymo Perception dataset.

HD-map handling: Waymo’s proto puts map_features only on the first frame of each segment. This processor caches the decoded world-frame map features per segment_id and applies the per-frame inverse pose on every frame. WaymoPerceptionDatasetConverter pre-populates the cache before parallel work begins, so both sequential and parallel preprocessing produce HD maps for every frame.

Parameters:

common_output_path (str)
split (str)
index_data_generator (IndexDataGenerator | None)
adapters (list[AbstractAdapter] | None)
context_aggregators (list[SegmentContextAggregator] | None)

CAMERAS_ORDER = {CameraDirection.FRONT: 1, CameraDirection.FRONT_LEFT: 2, CameraDirection.FRONT_RIGHT: 3, CameraDirection.SIDE_LEFT: 4, CameraDirection.SIDE_RIGHT: 5}¶

DATASET_NAME = 'waymo_perception'¶

property allowed_splits: list[str]¶: Return the list of allowed splits for the Waymo Perception dataset.

property context_aggregators¶

property dataset_name: str¶: Return the name of the dataset.

property inner_path: str¶: Return the inner path relative to the common output path.

needs_attr(attr)¶

Whether at least one registered adapter reads this StandardFrameData field. Used by per-dataset processors to skip expensive modality builds (cameras, lidar, hd_map, detections, …) when no adapter would consume them. True when attr is in the consumed-attrs union, plus a hard-coded special case: the identifier / index fields are always treated as needed since they are required for the cache + index regardless of adapter chain.

Return type:: bool
Parameters:: attr (StandardFrameDataField)

property output_path: str¶: Return the output path for the processed dataset.

process_frame(raw_frame_data)¶

Return type:: tuple[TransformedFrameData, FrameIndexData]
Parameters:: raw_frame_data (Any)

process_frame_and_save_data(raw_frame_data)¶

Process a single frame of raw data, save the processed frame data to disk, and return the corresponding FrameIndexData.

Return type:: FrameIndexData
Parameters:: raw_frame_data (Any)

property specific_output_path: str¶: Return the specific output path for the dataset.

property split: str¶: Return the dataset split.