Class UriCanonicalizer

java.lang.Object
ai.pipeline.connector.intake.util.UriCanonicalizer
public class UriCanonicalizer extends Object
Utility for canonicalizing URIs to create stable document IDs.

Transforms URIs into a normalized form that maximizes collision between different strings pointing to the same resource. Used for deterministic doc_id generation.

Canonicalization rules:

  • Validates URI syntax using Apache Commons Validator
  • Normalizes path navigation (../, ./)
  • Lowercases scheme and host only (preserves case in path/query)
  • Removes trailing slashes (except root "/")
  • Sorts query parameters alphabetically
  • Removes fragments (client-side only)
  • Normalizes default ports (:80, :443)

  • Constructor Details

    • UriCanonicalizer

      public UriCanonicalizer()

  • Method Details

    • canonicalizeUri

      public static String canonicalizeUri(String rawUri, boolean keepQueryParams)
      Transforms a source URI into a canonical ID string.

      The canonical form is designed to maximize collision between different URI strings that represent the same resource, while preserving case sensitivity where it matters (paths, query parameters).

      Parameters:
      rawUri - The input source URI
      keepQueryParams - If false, strips query parameters for a "base resource" ID
      Returns:
      A normalized string suitable for use as a unique identifier
      Throws:
      IllegalArgumentException - if the URI is invalid or cannot be parsed

    • canonicalizeUri

      public static String canonicalizeUri(String rawUri)
      Canonicalizes a URI, keeping query parameters. Convenience method that calls canonicalizeUri(String, boolean) with keepQueryParams=true.
      Parameters:
      rawUri - The input source URI
      Returns:
      The canonicalized URI string