Index (connector-intake-service 0.1.1 API)

A B C D E F G H I L M N P R S T U V
All Classes and Interfaces|All Packages|Serialized Form

A

accountId - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Account ID this connector belongs to.
addBytesProcessed(String, long) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Add the number of processed bytes to the session total.
addBytesProcessed(String, long) - Method in class ai.pipeline.connector.intake.service.SessionManager: Add the number of processed bytes to the session total.
ai.pipeline.connector.intake.entity - package ai.pipeline.connector.intake.entity
ai.pipeline.connector.intake.repository - package ai.pipeline.connector.intake.repository
ai.pipeline.connector.intake.service - package ai.pipeline.connector.intake.service
ai.pipeline.connector.intake.util - package ai.pipeline.connector.intake.util
ai.pipeline.connector.util - package ai.pipeline.connector.util
ApiKeyUtil - Class in ai.pipeline.connector.util: Utilities for API key generation and hashing.
ApiKeyUtil() - Constructor for class ai.pipeline.connector.util.ApiKeyUtil

B

bucketName - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.UploadResult
bytesProcessed - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Total bytes processed.

C

ChunkAssembler - Class in ai.pipeline.connector.intake.service: Service for assembling streaming chunks into complete documents.
ChunkAssembler() - Constructor for class ai.pipeline.connector.intake.service.ChunkAssembler
cleanup() - Method in class ai.pipeline.connector.intake.service.DocumentProcessor: Cleanup method to properly shut down the worker thread pool.
cleanupStaleChunks() - Method in class ai.pipeline.connector.intake.service.ChunkAssembler: Clean up stale chunk buffers that have not been updated recently.
cleanupStaleSessions() - Method in class ai.pipeline.connector.intake.service.StreamingChunkProcessor: Clean up stale streaming sessions.
completedAt - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Timestamp when session completed.
completeSession(String, String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Mark a session as completed and set its terminal state.
completeSession(String, String) - Method in class ai.pipeline.connector.intake.service.SessionManager: Mark a session as completed and persist its final state.
connectorId - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Connector ID that started this session.
ConnectorIntakeServiceImpl - Class in ai.pipeline.connector.intake.service: gRPC service implementation for Connector-Intake.
ConnectorIntakeServiceImpl() - Constructor for class ai.pipeline.connector.intake.service.ConnectorIntakeServiceImpl
ConnectorMetadata - Class in ai.pipeline.connector.intake.util: Utility class for parsing and serializing connector metadata JSON.
ConnectorMetadata() - Constructor for class ai.pipeline.connector.intake.util.ConnectorMetadata
connectorType - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Connector type identifier.
ConnectorValidationService - Class in ai.pipeline.connector.intake.service: Service for validating connector API keys and fetching connector configuration.
ConnectorValidationService() - Constructor for class ai.pipeline.connector.intake.service.ConnectorValidationService
contentHash - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.ProcessingResult
contentHash - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.UploadResult
CrawlDocument - Class in ai.pipeline.connector.intake.entity: Document tracking entity for orphan detection.
CrawlDocument() - Constructor for class ai.pipeline.connector.intake.entity.CrawlDocument: Default constructor for JPA.
CrawlDocument(String, String, String) - Constructor for class ai.pipeline.connector.intake.entity.CrawlDocument: Create a new crawl document tracking entry.
CrawlDocument.CrawlDocumentId - Class in ai.pipeline.connector.intake.entity: Composite primary key for CrawlDocument.
CrawlDocumentId() - Constructor for class ai.pipeline.connector.intake.entity.CrawlDocument.CrawlDocumentId
CrawlDocumentId(String, String) - Constructor for class ai.pipeline.connector.intake.entity.CrawlDocument.CrawlDocumentId
CrawlDocumentRepository - Class in ai.pipeline.connector.intake.repository: Repository for crawl document tracking operations.
CrawlDocumentRepository() - Constructor for class ai.pipeline.connector.intake.repository.CrawlDocumentRepository
crawlId - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Client-provided crawl ID.
CrawlSession - Class in ai.pipeline.connector.intake.entity: Crawl session entity tracking document ingestion sessions.
CrawlSession() - Constructor for class ai.pipeline.connector.intake.entity.CrawlSession: Default constructor for JPA.
CrawlSession(String, String, String, String, String) - Constructor for class ai.pipeline.connector.intake.entity.CrawlSession: Create a new crawl session.
crawlSessionId - Variable in class ai.pipeline.connector.intake.entity.CrawlDocument.CrawlDocumentId
CrawlSessionRepository - Class in ai.pipeline.connector.intake.repository: Repository for crawl session operations.
CrawlSessionRepository() - Constructor for class ai.pipeline.connector.intake.repository.CrawlSessionRepository
createSession(String, String, String, String, String, String, boolean, boolean) - Method in class ai.pipeline.connector.intake.service.SessionManager: Create a new crawl session.

D

deleteBySessionId(String) - Method in class ai.pipeline.connector.intake.repository.CrawlDocumentRepository: Delete all tracked document entries for the given session.
deleteOrphans - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Whether to delete orphans at end of crawl.
documentId - Variable in class ai.pipeline.connector.intake.entity.CrawlDocument: Repository document ID (UUID).
DocumentProcessor - Class in ai.pipeline.connector.intake.service: Service for processing individual documents.
DocumentProcessor() - Constructor for class ai.pipeline.connector.intake.service.DocumentProcessor
documentRef - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.ProcessingResult
documentsFailed - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Number of documents that failed processing.
documentsFound - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Number of documents found during crawl.
documentsProcessed - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Number of documents successfully processed.
documentsSkipped - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Number of documents skipped.

E

endCrawlSession(EndCrawlSessionRequest) - Method in class ai.pipeline.connector.intake.service.ConnectorIntakeServiceImpl: End a crawl session and persist final statistics.
equals(Object) - Method in class ai.pipeline.connector.intake.entity.CrawlDocument.CrawlDocumentId

F

findByAccountId(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Find all sessions for an account.
findByConnectorAndCrawl(String, String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Find a session by connector ID and crawl ID.
findByConnectorId(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Find all sessions for a connector.
findByDocumentId(String) - Method in class ai.pipeline.connector.intake.repository.CrawlDocumentRepository: Find all tracked entries referencing the given repository document ID across sessions.
findById(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Find a session by its unique identifier.
findBySessionAndSource(String, String) - Method in class ai.pipeline.connector.intake.repository.CrawlDocumentRepository: Find a tracked document by session and source ID.
findBySessionId(String) - Method in class ai.pipeline.connector.intake.repository.CrawlDocumentRepository: Find all tracked documents for a given session.
findByState(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Find sessions by state.
findStaleSessions(OffsetDateTime) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Find running sessions whose last heartbeat is older than the cutoff time.
fromJson(String) - Static method in class ai.pipeline.connector.intake.util.ConnectorMetadata: Parse from JSON string.

G

generateApiKey() - Method in class ai.pipeline.connector.util.ApiKeyUtil: Generate a new secure random API key.
getDefaultMetadata() - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
getMaxFileSize() - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
getRateLimitPerMinute() - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
getS3BasePath() - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
getS3Bucket() - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
getSession(String) - Method in class ai.pipeline.connector.intake.service.SessionManager: Retrieve a crawl session by its identifier.

H

hashApiKey(String) - Method in class ai.pipeline.connector.util.ApiKeyUtil: Hash an API key for secure storage.
hashCode() - Method in class ai.pipeline.connector.intake.entity.CrawlDocument.CrawlDocumentId
heartbeat(HeartbeatRequest) - Method in class ai.pipeline.connector.intake.service.ConnectorIntakeServiceImpl: Heartbeat endpoint for long-running crawls.

I

id - Variable in class ai.pipeline.connector.intake.entity.CrawlDocument: Composite primary key.
id - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Unique session identifier (primary key).
incrementDocumentsFailed(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Increment the counter of documents that failed processing for the session.
incrementDocumentsFailed(String) - Method in class ai.pipeline.connector.intake.service.SessionManager: Increment the count of documents that failed processing for a session.
incrementDocumentsFound(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Increment the counter of documents discovered for the session.
incrementDocumentsFound(String) - Method in class ai.pipeline.connector.intake.service.SessionManager: Increment the count of documents discovered for a session.
incrementDocumentsProcessed(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Increment the counter of documents successfully processed for the session.
incrementDocumentsProcessed(String) - Method in class ai.pipeline.connector.intake.service.SessionManager: Increment the count of documents successfully processed for a session.
incrementDocumentsSkipped(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Increment the counter of documents skipped (not processed) for the session.
incrementDocumentsSkipped(String) - Method in class ai.pipeline.connector.intake.service.SessionManager: Increment the count of documents skipped (not processed) for a session.
isComplete - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.ProcessingResult

L

lastHeartbeat - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Timestamp of last heartbeat.

M

metadata - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Crawl metadata as JSON.

N

nodeId - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.UploadResult

P

processChunk(StreamingChunk) - Method in class ai.pipeline.connector.intake.service.ChunkAssembler: Deprecated.
Use StreamingChunkProcessor with header/raw/footer protocol instead
processChunk(StreamingChunk, String, String, String, String, Map<String, String>) - Method in class ai.pipeline.connector.intake.service.StreamingChunkProcessor: Process a streaming chunk for a large file using header/raw/footer protocol.
processDocument(CrawlSession, ConnectorConfig, DocumentData) - Method in class ai.pipeline.connector.intake.service.DocumentProcessor: Process a document.
processedAt - Variable in class ai.pipeline.connector.intake.entity.CrawlDocument: Timestamp when document was processed.
ProcessingResult(boolean, StreamingChunkProcessor.UploadResult, String) - Constructor for class ai.pipeline.connector.intake.service.StreamingChunkProcessor.ProcessingResult
ProcessingResult(boolean, StreamingChunkProcessor.UploadResult, String, String) - Constructor for class ai.pipeline.connector.intake.service.StreamingChunkProcessor.ProcessingResult

R

RepositoryClient - Class in ai.pipeline.connector.intake.service: Client for repository-service integration.
RepositoryClient() - Constructor for class ai.pipeline.connector.intake.service.RepositoryClient

S

s3Key - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.UploadResult
SessionManager - Class in ai.pipeline.connector.intake.service: Service for managing crawl session lifecycle.
SessionManager() - Constructor for class ai.pipeline.connector.intake.service.SessionManager
setDefaultMetadata(Map<String, String>) - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
setMaxFileSize(long) - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
setRateLimitPerMinute(long) - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
setS3BasePath(String) - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
setS3Bucket(String) - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata
size - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.UploadResult
sourceId - Variable in class ai.pipeline.connector.intake.entity.CrawlDocument.CrawlDocumentId
sourceSystem - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Source system identifier.
startCrawlSession(StartCrawlSessionRequest) - Method in class ai.pipeline.connector.intake.service.ConnectorIntakeServiceImpl: Start a new crawl session using the unary RPC endpoint.
startedAt - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Timestamp when session started.
state - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Session state.
storeDocument(String, String, String, String, String, byte[], Map<String, String>) - Method in class ai.pipeline.connector.intake.service.RepositoryClient: Store a document in the repository-service.
streamDocuments(Multi<DocumentIntakeRequest>) - Method in class ai.pipeline.connector.intake.service.ConnectorIntakeServiceImpl: Stream processing entry point for the Connector Intake gRPC service.
StreamingChunkProcessor - Class in ai.pipeline.connector.intake.service: Processor for handling streaming chunks from large file uploads.
StreamingChunkProcessor() - Constructor for class ai.pipeline.connector.intake.service.StreamingChunkProcessor
StreamingChunkProcessor.ProcessingResult - Class in ai.pipeline.connector.intake.service: Result of processing a chunk.
StreamingChunkProcessor.UploadResult - Class in ai.pipeline.connector.intake.service: Upload result containing metadata about the uploaded file.

T

toJson() - Method in class ai.pipeline.connector.intake.util.ConnectorMetadata: Convert this metadata object to its JSON representation.
trackDocument(String, String, String) - Method in class ai.pipeline.connector.intake.repository.CrawlDocumentRepository: Track a document in a session.
trackDocuments - Variable in class ai.pipeline.connector.intake.entity.CrawlSession: Whether to track documents for orphan detection.

U

updateHeartbeat(String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Update the last heartbeat timestamp for a session to the current time.
updateHeartbeat(String) - Method in class ai.pipeline.connector.intake.service.SessionManager: Update a session's heartbeat timestamp to the current time.
updateState(String, String) - Method in class ai.pipeline.connector.intake.repository.CrawlSessionRepository: Update the persisted state for a session.
uploadResult - Variable in class ai.pipeline.connector.intake.service.StreamingChunkProcessor.ProcessingResult
UploadResult(String, String, long, String, String) - Constructor for class ai.pipeline.connector.intake.service.StreamingChunkProcessor.UploadResult

V

validateConnector(String, String) - Method in class ai.pipeline.connector.intake.service.ConnectorValidationService: Validate a connector's API key and fetch its configuration.
verifyApiKey(String, String) - Method in class ai.pipeline.connector.util.ApiKeyUtil: Verify an API key against a stored hash.

A B C D E F G H I L M N P R S T U V
All Classes and Interfaces|All Packages|Serialized Form