Class RealtimeToOfflineSegmentsTaskMetadata


  • public class RealtimeToOfflineSegmentsTaskMetadata
    extends BaseTaskMetadata
    Metadata for the minion task of type RealtimeToOfflineSegmentsTask. The watermarkMs denotes the time (exclusive) upto which tasks have been executed. This gets serialized and stored in zookeeper under the path MINION_TASK_METADATA/${tableNameWithType}/RealtimeToOfflineSegmentsTask PinotTaskGenerator: The watermarkMs> is used by the RealtimeToOfflineSegmentsTaskGenerator, to determine the window of execution for the task it is generating. The window of execution will be [watermarkMs, watermarkMs + bucketSize) PinotTaskExecutor: The same watermark is used by the RealtimeToOfflineSegmentsTaskExecutor, to: - Verify that is is running the latest task scheduled by the task generator - Update the watermark as the end of the window that it executed for
    • Constructor Detail

      • RealtimeToOfflineSegmentsTaskMetadata

        public RealtimeToOfflineSegmentsTaskMetadata​(String tableNameWithType,
                                                     long watermarkMs)
    • Method Detail

      • getWatermarkMs

        public long getWatermarkMs()
        Get the watermark in millis
      • toZNRecord

        public org.apache.helix.zookeeper.datamodel.ZNRecord toZNRecord()
        Specified by:
        toZNRecord in class BaseTaskMetadata
        Returns:
        ZNRecord containing the task metadata