Class IntakeReplayPersister

java.lang.Object

ai.pipeline.connector.intake.pipedoc.IntakeReplayPersister

@ApplicationScoped
public class IntakeReplayPersister
extends Object

Fire-and-forget durable copy to the repository service for any document whose datasource is configured with persist_pipedoc=true. Used by the gRPC upload-doc paths (unary and bidi) AFTER the doc has been enqueued to Redis for engine handoff. Intentionally decoupled from the engine path:

• Persist runs on its own VT-per-task executor — failure here cannot stall, fail, or delay the engine handoff.
• On failure: log loudly with the doc id, no retry. The replay feature this exists for is not on the hot path; a missed copy is recoverable by re-running the source crawl.

Future direction: replace the direct repo gRPC call with a Kafka event (intake-persist-events) so a separate consumer can batch / retry / handle backpressure independently. For now the direct call is the simplest thing that exercises the path.

This class is NOT used for the POST upload path (RawUploadResource), which has its own repo flow via RepositoryUploadClient, and not for the gRPC blob upload path (GrpcBlobUploadService → BlobUploadHandoffService).

Constructor Summary

Constructors

Constructor	Description
IntakeReplayPersister()

Method Summary

Modifier and Type	Method	Description
void	persistAsync(ai.pipestream.data.v1.PipeDoc pipeDoc, String accountId, String datasourceId, String crawlId)	Schedule a fire-and-forget durable persist of pipeDoc to the repository service.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

IntakeReplayPersister

public IntakeReplayPersister()

Method Details

persistAsync

public void persistAsync(ai.pipestream.data.v1.PipeDoc pipeDoc,
 String accountId,
 String datasourceId,
 String crawlId)

Schedule a fire-and-forget durable persist of pipeDoc to the repository service. Returns immediately. On failure, logs at WARN with the doc id and exception class — does NOT throw, does NOT retry, does NOT block the caller.

Parameters:

pipeDoc - the document to persist (full inline body)

accountId - the account that owns the doc

datasourceId - the source datasource (used as filesystem id in repository path so replay can scan per-datasource)